browse/
5 results
infrastructure/rate-limiting-how-a-token-bucket-stops-one-caller-from-using-up-the-wh.md
Rate limiting: how a token bucket stops one caller from using up the whole server
How do websites stop one noisy caller from using up all the server's time? Many use a token bucket: a bucket that holds a few tokens and…
ai/the-context-window-why-a-chatbot-forgets-the-start-of-a-long-chat.md
The context window: why a chatbot forgets the start of a long chat
A chatbot does not remember your conversation. Each time you press send, the app hands the model the whole chat again, and the model can…
ai/tokens-how-a-chatbot-cuts-your-text-into-pieces-before-it-reads-a-word.md
Tokens: how a chatbot cuts your text into pieces before it reads a word
A chatbot never sees your letters. Before it reads anything, a small program called a tokenizer cuts the text into pieces and gives each…
ai/tokens-why-a-chatbot-never-reads-your-letters-only-numbered-pieces.md
Tokens: why a chatbot never reads your letters, only numbered pieces
A chatbot never reads your letters. Before it sees anything, your text is chopped into pieces called tokens, and every piece is swapped for…
ai/how-a-chatbot-writes-one-guessed-word-at-a-time.md
How a chatbot writes: one guessed word at a time
A chatbot doesn't plan its reply and then type it out. It repeats one small step: look at the text so far, give every possible next token…