AI labs are intentionally making small models forget facts

New AI models are crushing complex math benchmarks while getting worse at simple trivia — and it's completely intentional.

Developer w4g1 notes that labs are intentionally stripping factual memory out of modern models to pack elite reasoning into smaller sizes. Models like DeepSeek V4-Flash run on just 13 billion active parameters per token, but factual recall without search tools is abysmal. Small Qwen3.5 models suffer 80% to 82% hallucination rates on knowledge benchmarks. Facts cost roughly two bits of memory per parameter to store in weights, and they go stale fast.

Why it matters: Instead of wasting parameters memorizing Wikipedia, labs are training models to act purely as logic processors. By letting search tools and local files supply the static facts at runtime, frontier-quality reasoning will soon run on a standard 24GB consumer GPU.

Know this: Reasoning procedures don't age, but hardcoded facts rot. Algebra works the same as it did decades ago, while code libraries and API documentation change constantly. Passing local docs directly to a model keeps the logic sharp and the answers accurate.

Turns out you don't need a data center when your model stops trying to memorize the internet.