AI Training vs. Inference: Where Is the Money Going?

By Adam Patti, Chief Executive Officer, VistaShares | Published November 13, 2026

Training is the process of building an AI model by running it through enormous amounts of data, while inference is the work of using a trained model to answer questions, generate content or take actions for users. For most of the AI boom, training drew the headlines, but spending is shifting: Gartner forecast that in 2026, inference would account for 55% of spending on AI-optimized cloud infrastructure, surpassing training for the first time. That shift affects which chips, data center designs and locations capture AI spending.

What is the difference between training and inference?

Training teaches a model by adjusting billions or trillions of parameters as it processes data. Large training runs use thousands of accelerators linked by high-speed networks for weeks or months, typically in a single campus. Training is episodic: a company trains a model, then moves on to the next one.

Inference happens every time someone uses a model. Each query, image or agent action requires computation. Inference is continuous and grows with the number of users and the complexity of their requests. It can run across many sites, closer to users, and on a wider range of chips.

A third category, often called post-training, has grown as well. It includes fine-tuning and reinforcement learning that refine a model after initial training, along with “reasoning” techniques that use more computation at the moment a question is answered.

How is spending shifting?

Gartner forecast that worldwide spending on AI-optimized infrastructure as a service would reach $42.3 billion in 2026, up 96% from 2025, with $23.3 billion supporting inference and $19.0 billion supporting training.¹ It projected inference’s share rising to 59% in 2027.¹ Deloitte projected that inference, the running of AI models, would make up about two-thirds of all AI compute in 2026.²

Usage data points the same way. Google reported processing about 9.7 trillion tokens a month across its products in May 2024; by May 2026, it reported more than 3 quadrillion a month.³ A token is a small unit of text or data that a model reads or writes, so token counts are a rough measure of inference volume.

Why is inference growing so quickly?

More users and applications. AI features are being built into search, office software, coding tools, customer service and consumer apps.

Agents. AI agents perform multistep tasks, calling a model many times for each request.

Reasoning models. Models that “think” before answering use more computation per query.

Longer inputs. Users increasingly send long documents, codebases and video, which require more processing.

Lower prices per token have also encouraged more use, a pattern economists sometimes compare to the Jevons paradox, in which efficiency gains lead to higher total consumption.

Why does the shift matter for the supply chain?

Chips. Training favors the most powerful accelerators with large memory and fast interconnects. Inference can run on a wider range of chips, which creates openings for custom designs and specialized inference processors. Our article on GPUs versus custom AI chips covers this competition.

Memory. Inference on large models depends heavily on memory bandwidth and capacity, which supports demand for high-bandwidth memory.

Data center locations. Training campuses can be built where power is cheap and plentiful. Some inference workloads benefit from being closer to users, which may favor sites near population centers and edge facilities.

Spending pattern. Training spending comes in large, lumpy investments. Inference spending recurs with usage, which some analysts see as a steadier source of demand over time.

How do costs differ between the two?

Training costs are largely fixed for a given run: a developer pays for a set amount of computing time, and the expense is incurred before the model earns any revenue. Inference costs scale with use. Each additional user or query adds a small incremental cost, which falls as chips become more efficient and software improves. For AI providers, that difference shapes pricing and margins. A popular model can spread its training cost across many users, but its inference bill rises with every new customer, so efficiency per query becomes a central business concern.

What are the uncertainties?

Forecasts of the training and inference split vary because companies do not report it consistently and the same hardware can be used for both. Efficiency gains could reduce the computation needed per query faster than usage grows, or usage could grow faster than expected. New training methods could again raise training’s share. Investors should treat precise percentages as estimates. Our article on hyperscaler capex shows how these workloads translate into capital spending.

What does this mean for investors?

The shift toward inference broadens the range of companies tied to AI demand, from chip designers and memory makers to data center operators, networking suppliers and power providers in more locations. It also introduces competition, as specialized inference hardware challenges incumbents. Funds focused on AI may hold companies exposed to training, inference or both, and holdings show which side a fund emphasizes.

Frequently asked questions

What is AI inference in simple terms?

Inference is using a trained AI model to produce an output. When someone asks a chatbot a question, asks software to summarize a document or uses an AI coding assistant, the model performs inference. Each request requires computing power, so inference costs grow with the number of users, the length of their requests and the complexity of the tasks.

Is training spending declining?

Not in absolute terms. Gartner projected training spending on AI-optimized cloud infrastructure of $19.0 billion in 2026, still growing. What is changing is the share: inference is growing faster, so training represents a smaller portion of the total. Frontier model developers continue to invest heavily in larger training runs and post-training methods.

What is a token?

A token is a small chunk of data, often a piece of a word, that an AI language model processes. Models read input tokens and generate output tokens, and cloud providers often price AI services per million tokens. Because each token requires computation, the number of tokens processed is a common, if imperfect, gauge of inference demand.

Do training and inference use different chips?

They can, but often they use the same ones. The most powerful accelerators handle both. Some companies run inference on older or less expensive chips, on custom designs built for their own workloads, or on specialized inference processors. The choice depends on cost per query, latency requirements, power availability and the size of the model being served.

Why do some people say inference spending is more durable?

Training investments come in large bursts tied to new model releases. Inference spending recurs every time the model is used, so it tends to rise with adoption rather than with research cycles. That pattern could make demand steadier, though it also depends on whether AI applications keep attracting users and whether prices per query keep falling.

How do thematic ETFs reflect this shift?

AI and semiconductor ETFs often hold companies tied to both training and inference. Some funds emphasize the biggest accelerator designers, which benefit from both. Others hold memory makers, networking firms, custom chip partners or data center operators whose exposure depends more on inference growth. Reviewing holdings and methodology shows how a fund is positioned.

Cite this page: Adam Patti, "AI Training vs. Inference: Where Is the Money Going?," VistaShares, updated November 13, 2026, https://www.vistashares.com/ai-training-vs-inference-spending/. Plain-text version.

This material is provided for educational purposes only. It is not investment, legal or tax advice, and it is not a recommendation or an offer to buy or sell any security. Information is drawn from sources believed to be reliable as of the date of publication, is not guaranteed as to accuracy or completeness, and may change without notice. Figures attributed to third parties are their estimates. Projections and forecasts are not guarantees of future results. Investors should consult their own financial, legal and tax professionals before making investment decisions. Investments involve risk, including possible loss of principal. Companies in a single industry or theme may be affected by common economic, regulatory or technological developments and may be more volatile than the broad market. References to companies, governments or programs are for illustration only.