6 Factors Government Agencies Should Consider

The true cost of AI encompasses a range of factors that go beyond just the initial investment in hardware and software.

John Mark Suhy

June 25, 2024 3:58 p.m.

3 minutes of reading

The promises of artificial intelligence encompass a wide range of expectations, both in terms of technological capabilities and the impact on the way we do business. However, government agencies should consider that the cost of AI can be multifaceted and extend beyond the immediate monetary value. AI will both increase and reduce costs. It is therefore important to consider an investment in AI from a holistic perspective.

Here are some key factors to consider:

Hardware costs: Graphics processing units (GPUs) are fundamental to the advancement of artificial intelligence and form the backbone of AI innovation. The availability of these critical components is currently hampered by supply shortages, contributing to significant cost increases.
Energy costs: Training complex AI models requires significant computing power, which in turn consumes a considerable amount of energy. The energy costs associated with operating the underlying infrastructure to power this can be substantial.
Multi-Agent Costs: Multi-agent generative AI frameworks are essential for advancing generative AI. They make more extensive use of the underlying large language models (LLMs), leading to increased computational demand and costs compared to using a single LLM such as GPT 4.
Data Acquisition and Management: Whether you are fine-tuning a basic generative model or creating your own models, high-quality data is crucial to training AI models and helping resolve issues like “drift.” Acquiring and maintaining large data sets can be expensive, as can the ongoing costs associated with storing, processing, and managing the data. The old adage “junk-in-junk-out” is a key consideration here.
Personnel costs: Skilled personnel such as data scientists, machine learning engineers, and AI researchers are currently essential to develop, integrate, and maintain AI systems. These professionals often earn high salaries, which can represent a significant ongoing expense. Experience is essential because training or fine-tuning models can be extremely expensive, and errors requiring rework can add up quickly.
Ethical and regulatory costs: Complying with ethical guidelines and regulatory requirements can add additional costs to AI projects. This may include ensuring data privacy, addressing bias and fairness issues, and complying with industry-specific regulations. The rules on this are still being developed: only recently has the US government provided guidance on protecting AI, and implementing these safeguards is going to cost a lot of money, just like Zero Trust and other initiatives.

The true cost of AI encompasses a range of factors that go beyond just the initial investment in hardware and software. GPU shortages and intense compute demands can further inflate these costs, highlighting the importance of careful planning and resource management in AI projects. However, if done correctly, the benefits that AI will bring will far outweigh the costs in the long term.

Today’s AI landscape reflects the tumultuous early days of the Wild West – a time of exploration and untapped potential. As the U.S. government crosses this new frontier, costs associated with AI projects are expected to be historically high, reminiscent of the early days of the Internet or the space race. During these periods, initial investments were substantial because the technologies were in their infancy, standards were non-existent, and the path forward was unclear. As AI policies, standards, and best practices are developed and refined, costs will likely normalize, but this initial phase of high spending is a critical step toward harnessing AI’s transformative potential. AI for the public good.

John Mark Suhy is Chief Technology Officer of Greystones Group.