Fill in the Blank (What Have Language Models Learned?)
A visual explainer from Google PAIR that probes what the language model BERT learned about the world by asking it to fill in a blank. Change a single word, as in 'in texas, they like to buy ___', and you can watch the candidate words and their probabilities reorder themselves. You come away understanding that a language model's predictions are statistical associations drawn from the text it read, not neutral knowledge, and that social bias is learned along with everything else.
How to try it
Click a word in one of the example sentences, or rewrite the sentence in the input box and press 'Update Sentence', and BERT's candidate words appear in a list with their probabilities. The comparison tool below lets you choose how many tokens to show and which chart type, and switch the model between BERT and Zari to compare predictions for two sentences. The page is English only, and as it notes itself, the text the model was trained on includes harmful stereotypes.