How LLMs Work

Overview

Effective use of LLM-based tools such as Claude Code requires a foundational understanding of their internal mechanisms. This lesson examines three core aspects of Large Language Models: tokenization, probabilistic generation, and context windows.

Definition and Operating Principle

A Large Language Model is a probabilistic system that predicts the next token given an input sequence. Trained on large-scale text corpora, the model reproduces statistically likely continuations rather than performing semantic comprehension. The fundamental operation is pattern matching, not reasoning in the human sense.

Tokenization

LLMs process text in units called tokens. In English, one token corresponds to approximately 3–4 characters; the word "understanding" is typically segmented into "under" + "stand" + "ing". In Japanese, a single character often maps to 1–2 tokens.

Token count directly affects both cost calculation and context window consumption. Managing input volume is therefore a practical concern in production workflows.

Probabilistic Generation

LLMs do not retrieve answers from a database. Instead, they generate output one token at a time, computing at each step: "Given the sequence so far, which token has the highest probability of occurring next?"

A temperature parameter controls the degree of randomness in token selection. Consequently, the same prompt may yield different outputs across invocations.

Context Window

The context window defines the total number of tokens a model can reference in a single inference pass. This includes the prompt, conversation history, and any shared files. The Claude 4 family supports a context window of 200K tokens. However, as discussed in subsequent lessons, output quality begins to degrade well before this upper bound is reached.