In 2018, a Google team released BERT, and natural language processing was not the same afterward. If GPT is the model that made AI write, BERT is the model that made AI understand.
The key idea
Earlier language models read text in one direction — left to right — predicting the next word. BERT’s insight was to read both directions at once. During training it randomly hides (“masks”) words in a sentence and asks the model to fill them in using the words on both sides. To do that well, the model must build a rich understanding of context.
Why it mattered
BERT set new records across a broad suite of language-understanding benchmarks at the time, often by wide margins. More importantly, it cemented a recipe: pretrain a large Transformer on huge amounts of text, then fine-tune it cheaply for specific tasks. That pattern is now the foundation of the field.
Its lasting influence
BERT and its many variants became workhorses in production — powering search ranking, text classification, and information extraction at scale. When you get better search results or automatic content moderation, a BERT-family model is often quietly involved.
Who should read the original
Practitioners building understanding-focused NLP systems will find it foundational. For everyone else, the takeaway is that “read both directions and predict the missing word” was a deceptively simple idea with enormous consequences.