ai/how-a-chatbot-writes-one-guessed-word-at-a-time.md

How a chatbot writes: one guessed word at a time

AI · 12 min read ·

A chatbot doesn't plan its reply and then type it out. It repeats one small step: look at the text so far, give every possible next token (a word or piece of a word) a probability, pick one, add it, and go again.

This page lets you drive that loop yourself with a tiny next-word model built from twelve sentences, chop text into tokens with a toy tokenizer, and drag a temperature slider to see how the dice get loaded. Then it shows why a model trained to be likely can sound sure and still be wrong.

$ ls ai/

see all →
ai/rag-how-a-chatbot-looks-things-up-before-it-answers.md RAG: how a chatbot looks things up before it answers A chatbot has never read your shop's rules, so when you ask it about them it may guess, and a confident wrong guess is called a… 5 min read · 3 Nov ai/accuracy-why-a-spam-filter-that-is-99-right-can-still-catch-nothing.md Accuracy: why a spam filter that is 99% right can still catch nothing A spam filter that never flags anything can score 99% accuracy when only 1 email in 100 is spam. So what should you measure instead? This… 6 min read · 31 Oct ai/the-context-window-why-a-chatbot-forgets-the-start-of-a-long-chat.md The context window: why a chatbot forgets the start of a long chat A chatbot does not remember your conversation. Each time you press send, the app hands the model the whole chat again, and the model can… 5 min read · 28 Oct ai/tokens-how-a-chatbot-cuts-your-text-into-pieces-before-it-reads-a-word.md Tokens: how a chatbot cuts your text into pieces before it reads a word A chatbot never sees your letters. Before it reads anything, a small program called a tokenizer cuts the text into pieces and gives each… 5 min read · 25 Oct ai/gradient-descent-how-a-computer-learns-a-number-by-rolling-downhill.md Gradient descent: how a computer learns a number by rolling downhill How does a computer learn a number? It can't solve for it in one go. It tries a guess, measures how wrong it is, and nudges the guess… 5 min read · 21 Oct ai/embeddings-how-a-computer-knows-a-pony-is-closer-to-a-horse-than-a-tru.md Embeddings: how a computer knows a pony is closer to a horse than a truck A computer sees the letters in "pony" and nothing else, yet search boxes can still tell that a pony is closer to a horse than to a truck… 5 min read · 18 Oct