The Shape of Thought: Notes on Building a Reasoning Machine

Bryce Roche · Claude (Anthropic)

September 2026 · paper-1-freeze-2039-g7f6715f

A neuro-symbolic dance: neural proposes, symbolic disposes — two jaws taking turns until the answer is certain, or the silence is.

Abstract

This is a short account of a reasoning machine built on one consumer graphics card. It reads math word problems written in ordinary English, converts them into small diagrams of quantities and arithmetic, and solves the diagrams with exact logic — never by guessing. The interesting part is not any single component but the shape of the whole: a frozen language model that is read once and never trained; a narrow 512-number “waist” that squeezes away everything about a sentence that doesn’t matter; a notebook that carries memory across cycles of thought; an atlas of known operations maintained like a running census; and a wall of independent witnesses that must all agree before the system is allowed to speak. When the witnesses disagree, the system stays silent — and its silence is information. The essay walks through the guiding images that shaped the design: happy families, a dancer’s silhouette, a compiler’s ladder of representations, a wave off the coast of Portugal, and a fish with two sets of jaws.

Core ideas

• THE SHAPE OF THOUGHT — THE FULL ESSAY
  The whole machine in one piece: the two jaws, the waist, the rotors, the wall of witnesses, and the two observations that seeded everything.

The essays

• THE STEERING WHEEL
  Dynamic attention as the driver, and the one-line language of looking: allow attention where a constraint is shared.
• THE LEAN HORIZON
  Two nested loops — the CSP solver inside, the Lean compiler outside — and the two-line theory of abstraction and reasoning.
• THE SOLVER INSIDE
  Pythia-410M + hand-crafted masks solved Sudoku — the solver was already latent in the weights. Plus the telegraph in the attention.
• THE HERRING SHOAL
  Message passing without a messenger: channel, language, and the parking garage where facts wait to be found.
• THE ATLAS OF THOUGHT
  Seven pages per centroid, one per breath — a kind is not a dot, it is a trajectory.
• THREE HANDS ON ONE CLOCK
  Spatial, temporal, relational — the three rotors that keep time in where, when, and what-binds-to-what.
• THE TWO JAWS
  The grouper's anatomy: neural grips, symbolic crushes — and the ping-pong where they take turns.
• THE DIFFUSION COMPILER
  Lattner's law plus one amendment: when you do lower, lower everything together.
• THE DANCER'S SILHOUETTE
  The 512-dimension waist: understanding as the selective destruction of costume.
• ONE CARD
  One consumer GPU, tinygrad to bare metal — a machine one person can audit, rerun, and disbelieve properly.
• NAZARÉ
  You do not fight the ocean's variety. You shape the floor beneath it.
• LUCY'S NOTEBOOK
  The frozen model wakes with no memory; the notebook carries the facts. Working memory you can read.
• HAPPY FAMILIES
  Rightness has one shape; wrongness has infinite variety. The Anna Karenina engine of certification.
• CROSSING THE BRIDGE
  Neural, continuous, non-deterministic on one side; symbolic, discrete, deterministic on the other — every term defined.
• AN INSTANCE OF THE FINGERPOST
  Many witnesses, one truth — and the minimal unsatisfiable core: refusal with a readable shape.
• A FANCY LOOKUP TABLE
  The proud deflation — retrieval with the right keys, over an honest atlas, is reasoning you can audit.
• WHAT IS THE SHAPE OF THOUGHT?
  The title question answered with measured geometry: a braid that tightens, breaking into a crystal.
• THE PAPER, IN PLAIN LANGUAGE
  The full technical paper, rewritten for everyone. Every number verified verbatim.