Lesson 1 — What AI Assistants Actually Are

Hands-on · about 8 minutes.

Begin with the activity below, before reading the explanation. For each phrase, select the word you expect to come next.

This activity needs JavaScript enabled. The lesson below still covers everything.

What the activity demonstrated

You examined a piece of text and selected the most plausible next word. This is precisely the task that an AI assistant performs.

Claude, ChatGPT, Gemini, and Copilot are all large language models (LLMs). An LLM is a program trained on a very large quantity of text. From that text it has learned the statistical patterns of language sufficiently well to continue any passage in a manner that is generally coherent, relevant, and accurate.

This is the core mechanism: given the text so far, the model predicts what should follow. It does not retrieve facts from a database, nor does it perform deterministic computation as a calculator does. It generates the most plausible continuation. In most cases the most plausible continuation is also the correct one — but not in all cases, and that distinction is the single most important concept in this lesson.

How a model reads text — tokenization

Before a model can predict anything, it must convert text into a form it can compute on. It does this through three steps in sequence.

Step 1 — Tokenization. The model does not operate on individual characters or whole words. Its input units are tokens — contiguous spans of text that are typically a full word, or in some cases a sub-word fragment (e.g. token and ##izing). The input text is first split into this sequence of tokens.

Each token is then mapped to an integer called its vocabulary ID. A typical model vocabulary contains roughly 50,000 entries — a fixed list built before training begins. Every token the model will ever encounter has one unique number in that list:

"the" → ID 21 "dog" → ID 5 "jumps" → ID 49

So the sentence "the dog jumps" becomes the integer sequence [21, 5, 49]. The vocabulary ID is simply the token's address in that list — nothing more. Rare words that are not in the vocabulary are split into sub-word pieces that are, so any text can always be represented.

Step 2 — Embedding. Each vocabulary ID is used to look up a row in the model's embedding matrix — a large table learned during training. The row for ID 5 ("dog") is a list of roughly 768 numbers that encodes where dog sits in the model's internal meaning-space: closer to cat and puppy, far from logarithm. This vector is not hand-crafted; it was discovered automatically from patterns in the training data.

Step 3 — Attention. Once every token is an embedding vector, the model passes the entire sequence through many layers of a mechanism called self-attention. Each token's vector is continuously adjusted by the vectors of every other token in the context — so the meaning of bank shifts depending on whether river or money appeared nearby. After many such layers, the model uses the final adjusted vectors to compute a probability distribution over the entire vocabulary, and the highest-probability token becomes the next word it generates.

These three steps — tokenize, embed, attend — repeat for every token the model produces. The vocabulary ID is the entry point: the raw integer that unlocks the right embedding vector and sets the entire process in motion.

Where AI assistants are reliable

For each task below, determine whether an AI assistant performs it reliably. Feedback is provided immediately for each response.

This activity needs JavaScript enabled. The lesson below still covers everything.

Why these failures occur

The tasks at which an AI assistant performs poorly are not arbitrary; they share a common cause. A model predicts plausible text but does not verify that text against reality. It therefore fails in predictable ways:

Confident mistakes. A model can state something false in exactly the same calm, fluent tone it uses for the truth. There is no built-in "I am unsure" signal you can rely on.
Made-up details. Functions, citations, or settings that sound real but do not exist. This is called a hallucination.
Stale knowledge. Training data has a cutoff date, so a model may not know about recent changes — including recent changes to your own project.
Exact arithmetic and counting. Improving, but still a weak spot. Verify numbers.

These limitations do not render the tools unreliable; they define how the tools should be used. The model provides speed and breadth; the user provides verification and judgment.

The principal AI assistants

Tool	Made by	Typical use
Claude	Anthropic	Long documents, careful reasoning, and coding.
ChatGPT	OpenAI	General-purpose assistant, widely used.
Gemini	Google	General-purpose, integrated with Google products.
Copilot	GitHub / Microsoft	Code completion inside an editor.

They differ in specifics, but the conceptual model presented in this lesson applies to all of them. A thorough understanding of one transfers directly to the others.

Quick check

This activity needs JavaScript enabled.

Summary: an AI assistant predicts the most plausible next text — highly effective for drafting and explanation, but capable of being confidently incorrect, so every response requires human verification.

Next: How AI Predicts the Next Word →