Foundation Model

For most of the history of machine learning, you built a separate model for each job: one to translate text, another to spot tumours, another to recommend films. Each was trained from scratch on data for that single task.

A foundation model takes a different route. You train one very large model on a vast, broad pool of data, so it picks up general patterns about language, images, or even the structure of proteins. That single model can then be adapted, through a lighter round of training called fine-tuning, to do many specific jobs. The same underlying model might power a chatbot, a coding assistant, and a search tool.

The name, coined by researchers at Stanford in 2021, captures the idea: these models are a foundation you build on top of. Large language models are the most familiar example, but the pattern now extends to images, audio, and scientific domains such as predicting how molecules fold and interact.

The upside is that one expensive round of training can serve countless applications. The downside is concentration: if a flaw or bias sits in the foundation, every application built on it can inherit the same problem.