Systems that reproduce the prejudices in their data: a sentencing tool that scored Black defendants as higher risk, a hiring model that learned to penalise the word 'women's', face recognition that failed on dark-skinned faces. The term became mainstream with a 2016 investigation and a 2018 study, and it was the AI-ethics conversation before the chatbots arrived and made everyone talk about extinction instead.
Sighted in October 2018 as a Reuters report that a large retailer had scrapped an experimental recruiting model after discovering it downgraded CVs containing the word 'women's', having learned from a decade of mostly male hires. The model had done what it was asked.
Bias in this sense is not the statistical term, an estimator's systematic error, though the two get merged in arguments. It is the social one: a systematic disadvantage to a group. A model can be unbiased in the first sense and deeply biased in the second, and the confusion has cost several panels an afternoon.
I remember 2018, when this was the entire AI ethics conversation and the researchers doing it were the field's conscience. By 2023 they were being told their concerns were 'near-term' by people worrying about the end of the world, and the word had become a way of saying 'not the important risk'.
The record's definition: the model is not biased the way a person is. It is a mirror of the data, held up to a decision. If the past was unfair, the model learns the unfairness and applies it at scale, with a confidence score. The 2016 investigation's contribution was showing the mirror to the people it was pointed at.
The record is wrong. Show what it should say, and why.
