One card

Everything described elsewhere in this project — the frozen language model read once per problem, the narrow information waist, the three-rotor clock keeping track of where and when and what-connects- to-what, the wall of independent witnesses that has to agree before the system will speak — all of it runs on one piece of hardware: a single AMD 7900 XTX, a consumer gaming graphics card with 24 gigabytes of memory. Not a data-center cluster. Not a rack. One card, sitting in one machine, that anyone could buy.

It runs through tinygrad, a deliberately small machine-learning library. Most of the field runs on a handful of much larger, much more heavily engineered frameworks, but tinygrad takes the opposite bet: keep the codebase small enough that one person can actually read all of it, and have it compile its own GPU kernels directly rather than depending on a vendor’s enormous, opaque software stack sitting between the code and the chip. This matters specifically for AMD hardware, which has historically been harder to get solid machine-learning performance out of than the more commonly supported alternative — tinygrad drives the card well through a comparatively lean, custom-built layer of software, no vendor bloat required to get there. Our machine doesn’t even use the vendor’s driver stack: tinygrad speaks to the GPU through its own minimal driver, talking nearly bare metal — the shortest path between a line of Python and a transistor that we know of in machine learning.

What that buys became vivid this week, when we built the most demanding piece of software in the project: a training engine that splits the model’s seven-cycle deliberation loop into segments, runs an exact symbolic solver between them, and then walks backwards through all seven segments recomputing and accumulating gradients — sixteen separately captured GPU programs choreographed around a CPU that keeps interrupting with deductions. In most frameworks this fights the machinery. In tinygrad, each captured segment is traced once and compiled — through tinygrad’s own scheduler and code generators — into custom fused kernels for this exact card, replayed thereafter with essentially zero dispatch overhead and bit-identical determinism. When we needed to verify the whole seven-segment walk against the original fused graph, the two agreed to within one floating-point ulp forward and matched all 78 parameter gradients backward — because the entire stack, from the deliberation loop down to the kernel that runs it, is transparent enough to check.

That is the deeper reason this project runs on tinygrad. Small enough to read means small enough to trust; compile-your-own-kernels means the machine’s honesty extends all the way down. A certification architecture built on an inspectable substrate — turtles, for once, all the way down.

The scale of the actual learned part of the system is easy to understate if you don’t say it plainly. Underneath the whole machine sits a large frozen language model, and “frozen” is not a figure of speech — its weights never change, not once, over the entire course of this project. All of the task-specific intelligence the system has — everything it has learned about reading math word problems specifically — lives in a much smaller trained head sitting on top of that frozen giant, and that trained head amounts to a fraction of one percent of the system’s total parameters. The overwhelming majority of the numbers in this system never move. The frozen model does its heavy lifting exactly once per problem, hands off its reading to the small trained head, and that’s the entire extent of its involvement.

It would be easy to frame all of this — one card, one small library, a trained head that’s a rounding error against the frozen model beneath it — as a story about frugality, about doing more with less because that’s what was available. That’s not really the point, though it’s a true side effect. The actual reason to build this way is that a system whose every single experiment fits on one card is a system one person can fully audit. Every training run, every ablation, every claim in this project’s ledger can be rerun, from scratch, by the same person who ran it the first time, on the same desk, without needing anyone else’s cluster time or anyone else’s permission. A result that can’t be rerun cheaply is a result you eventually have to take on faith. Nothing in this project is supposed to be taken on faith. One card means nobody, including us, has an excuse not to disbelieve a claim and go check it.

Honesty about a system built this way requires being honest about what’s still missing, and there’s a real list. A perceiver — a dedicated organ meant to watch and segment raw input before any parsing happens — was explored earlier in this project and set aside as not yet earning its place; we are only now beginning to suspect it might belong back in the body after all. The atlas of known operation-shapes currently lives in ordinary flat space, but the kinds and sub-kinds it needs to represent have a tree-like structure that flat space fits poorly — a Poincaré ball, a curved hyperbolic space where trees embed naturally without distortion, is the likely future home for it, though it isn’t built yet. And the mid-thought conversation between the neural and symbolic jaws, described elsewhere as the ping-pong, is built and verified but still awaits its verdict on real, wild prose rather than the synthetic data it was trained on.

None of these get to skip the wall just because they sound promising. The project keeps a ledger recording exactly what each of these ideas has to demonstrate before it’s allowed to become part of the deployed system, and most ideas that get proposed never clear that bar. That’s not a flaw in the process. It’s the entire point of building the wall in the first place — a system this auditable doesn’t get to make exceptions for its own good ideas.

One card, one library, a trained head smaller than a rounding error, and a ledger that tells the truth about what hasn’t earned its place yet. That’s not a small machine pretending to be modest. It’s the only kind of machine you can actually trust.