NLP

BERT

Google's breakthrough AI model that reads sentences in both directions at once to understand context better.

Definition

Bidirectional Encoder Representations from Transformers, a language model Google introduced in 2018. Unlike earlier models that read text left to right, BERT reads an entire sentence in both directions at once, so it understands each word from the full surrounding context. It is pre-trained using masked language modeling, where random words are hidden and the model learns to predict them, then fine-tuned for specific tasks.

Why it matters

BERT reset the bar for language understanding and still underpins Google Search ranking and many production NLP systems. For teams, it means accurate sentiment analysis, classification, and question answering can be built by fine-tuning an existing model instead of training one from scratch.

Related terms in NLP

Chain of Thought (CoT)

Asking an AI to "show its work" and think step-by-step, which makes it much better at solving math and logic problems.

Context Window

The maximum amount of text an AI can read and consider at one time, like how many pages of notes it can hold in its head.

Conversational AI

AI that can have natural back-and-forth conversations with humans, chatbots, voice assistants, and customer service bots.

Embeddings

Converting words, images, or data into lists of numbers that capture their meaning, so similar things are mathematically close together.

Back to the full glossary

From vocabulary to outcomes

Ready to put BERT to work?

Knowing the term is step one. Deploying it inside a revenue architecture that compounds is what Sophizo builds.

Book a Discovery Call