How Large Language Models (LLMs) Work — Without the Maths

You do not need to understand transformers or attention maths to use an LLM well. But knowing four ideas — tokens, training, the context window and temperature — will explain most of the behaviour you see, including the frustrating parts.

1. Tokens: how the model reads

An LLM does not read letters or whole words. It reads tokens, which are small chunks of text. Common words like "the" are one token; a rare word like "Thiruvananthapuram" might be split into several. A rough rule for English: 100 tokens is about 75 words.

Why this matters to you:

• Pricing and limits for AI tools are counted in tokens.
• Models are sometimes bad at counting letters in a word — they never actually "see" individual letters.
• Languages like Hindi or Tamil often use more tokens for the same sentence than English does.

2. Training: two stages

Pre-training. The model reads an enormous amount of text and learns to predict the next token. This is where it picks up grammar, facts, coding patterns and general knowledge. It is extremely expensive and happens rarely.

Fine-tuning. The raw model is then trained further on examples of helpful conversations, and people rate its answers so it learns to be helpful, honest and safe. This is why a chatbot answers your question instead of just continuing your text like an autocomplete.

3. The context window: the model’s working memory

The context window is how much text the model can consider at once — your messages, its replies and any documents you paste. Modern models can handle hundreds of pages, but it is still a limit.

Two practical effects:

• In a very long chat, early details can get less attention. If the model "forgets" an instruction, repeat it or start a fresh chat with a summary.
• The model has no memory between separate chats unless the tool has a memory feature. Each new chat starts fresh.

4. Temperature: creativity vs consistency

At every step the model has a list of possible next tokens with probabilities. Temperature controls how adventurous it is when picking one.

• Low temperature — picks the most likely token almost every time. Consistent, predictable. Good for code, facts and data extraction.
• High temperature — sometimes picks less likely tokens. More varied and creative. Good for brainstorming and stories.

This is also why you can ask the same question twice and get two different answers.

Putting it together: what happens when you press Enter

1. Your message (plus the chat so far) is split into tokens.
2. The model processes all of them and calculates probabilities for the next token.
3. One token is chosen (temperature affects this), added to the answer, and the process repeats.
4. It stops when it predicts an end-of-answer token or hits a length limit.

Notice what is missing: there is no step where the model looks up the truth in a database. It generates text that fits the patterns it learned. That single fact explains hallucinations, which is the next lesson.