Getting a model to do a task by showing it a handful of examples in the prompt, with no training at all. The GPT-3 paper in 2020 put the phrase in its title and showed that a big enough model would pick up a new task from three or four examples, which nobody had expected. Zero-shot is the same with no examples. The discovery is why prompting became a job.
Sighted in every prompt guide since 2021 as the instruction to 'give it a few examples', usually with a note that three is better than one and ten is better than three, until the context runs out. The record notes that the advice is the 2020 paper's main figure, redrawn as a tip.
'Shot' does not mean an attempt. It means an example given, borrowed from one-shot learning in vision, where a system had to recognise a class from a single image. A zero-shot model has been given no examples, not no chances. The record notes that this confuses new readers roughly every week.
The record's definition of the surprise: before 2020, teaching a model a new task meant thousands of labelled examples and a training run. After, it meant typing 'English: cat. French: chat. English: dog. French:' and reading the answer. The paper called it in-context learning.
I remember the 2020 paper's arithmetic charts, where accuracy on two-digit addition went from nothing to nearly perfect as the model got bigger, with no training on sums. I thought it was a trick of the test set. It was not. It was the first time I believed the scaling people, and I did not tell them.
What does it mean? Write it the way you would say it out loud.
