Inference
Inference is the act of using a trained AI model to produce an answer. Training is the one-off, expensive process of teaching a model; inference is what happens every time afterwards when you actually ask it something and it responds. Because a popular model runs inference millions of times a day, it is the ongoing, recurring cost of operating an AI system, as opposed to the upfront cost of building it.
It helps to split an AI model’s life into two very different phases. First comes training, where the model learns by working through enormous amounts of data, adjusting its internal settings until it performs well. This is slow, costly, and done relatively rarely. Then comes inference, where the finished model is put to work: you type a question, it runs its calculations, and out comes an answer. Every chatbot reply, every generated image, and every fraud check is an act of inference.
A simple comparison is passing your driving test versus actually driving. The test is a demanding, one-time hurdle, much like training. Once you have passed, every individual journey afterwards is inference: quicker and cheaper than the test, but something you do again and again for years. The lifetime effort is dominated not by the test but by all those everyday trips.
This distinction has real business consequences, which is why it comes up so often in AI news. Training grabs the headlines because of its jaw dropping costs, but for a company running a live product, inference is frequently the larger bill over time, since it scales with every user and every request. Much of the current push for faster chips, cheaper hardware, and more efficient models is really a push to bring down the cost of inference so that AI can be offered affordably at scale.