LLM Basics Lesson 3 of 3

Hallucination

Overview

Hallucination refers to the generation of plausible-sounding but factually incorrect output. It is an inherent property of probabilistic language models and must be understood as a systematic risk rather than an occasional anomaly.

Causal Mechanism

LLMs predict the most likely next token based on statistical patterns. The model has no internal representation of truth or falsity — it generates sequences that are statistically plausible given the training distribution.

Implication: When the model lacks information to produce a correct answer, it does not abstain. It generates the most statistically plausible completion, which may bear no relation to factual accuracy.

Common Hallucination Categories

  • Fabricated APIs: Methods or functions that do not exist but conform to naming conventions of real libraries
  • Fictitious citations: References to papers, documentation, or specifications that were never published
  • Incorrect file paths: Plausible-sounding paths to files or directories absent from the codebase
  • Confident factual errors: Incorrect statements delivered with high confidence markers

Detection Indicators

The following patterns warrant verification:

  • Highly specific details (names, version numbers, paths) not present in the provided context
  • Claims about the codebase made without prior file reading operations
  • API references or library functions not encountered in the project dependencies
  • Confident assertions on inherently uncertain topics

Verification heuristic: Any specific information generated by the model that was not present in the input should be independently verified. This applies particularly to file paths, function signatures, and external references.

Mitigation Strategies

  • Provide source material: Supply actual code rather than descriptions to ground the model's output in real data
  • Request verification: Instruct the model to confirm file existence before modification
  • Constrain task scope: Smaller, well-defined tasks reduce the opportunity for fabrication
  • Require evidence: Request that the model cite specific code locations for its claims

Claude Code's tool-based architecture provides a structural mitigation: tools such as Read and Glob access the actual filesystem, grounding the model's reasoning in observed data rather than statistical interpolation.

Key Takeaways

  • Hallucination is a systematic property of probabilistic generation, not an anomaly
  • The model generates statistically plausible completions regardless of factual accuracy
  • All model-generated specifics not present in the input require verification
  • Tool-based file access provides structural hallucination mitigation