A model trained on a vast general corpus that other things are then built on top of: a base layer, hence the building metaphor. Stanford coined the term in a 200-page report with over a hundred authors in 2021 and founded a centre with the name in the same week, which some researchers found presumptuous. It became the term regulators used, including in the EU's AI Act.
Sighted in the European Parliament's 2023 amendments to the AI Act, which added obligations for 'foundation models' by name. The record notes that a phrase invented in a Californian report in August 2021 had become law in Brussels before it had become standard in the labs.
Filed with the building metaphor intact and its consequence noted: foundations are what everything else depends on and what nobody can see. The report's argument was that a flaw in the base propagates to every product above it. This is also true of buildings, and is why they have inspectors.
I remember the 2021 report being dismissed on Twitter as a hundred authors describing a thing that GPT-3 had already been for a year. Two years later, when I needed one word for the category in a document a lawyer would read, it was the only one available.
The coinage was contested at its own launch workshop, where senior researchers argued that the models were not foundations of anything yet and that Stanford was naming a field so as to own it. The term won anyway, partly because 'large pre-trained model' has no metaphor in it.
What does it mean? Write it the way you would say it out loud.
