Does a Small Model Have a Global Workspace?

In July, Anthropic published one of the most striking interpretability papers I have read: Verbalizable Representations Form a Global Workspace in Language Models. The claim is that a language model carries a small, privileged set of internal directions, the J-space, that works like the global workspace from consciousness science: a narrow broadcast channel holding whatever the model is currently poised to say, sitting on top of a vast ocean of automatic processing it cannot report.

The experiment in that paper I could not stop thinking about: delete the workspace, at every token, mid-thought, and the model keeps talking fluently. Shallow skills survive. Multi-step reasoning collapses. Unless the model is allowed to write its reasoning down, in which case it barely gets hurt. The page substitutes for the head.

Does any of this survive on a model 100x smaller?

The paper runs on frontier models (Sonnet 4.5, Opus). Its own limitations section admits nobody knows whether small models have “an equally rich workspace, a proportionally smaller one, a less reliable one, or none at all.” So I took their ablation protocol to Qwen3-4B and found out. Short version: the fourth option, with a twist.

The intervention, in one figure

The model’s working state at each token is a single vector, the residual stream. The paper’s Jacobian lens assigns every vocabulary word a direction in that space; leaning along the spider direction means the model is silently poised to say something spider-related. The ablation finds the ten directions the vector leans along most strongly and subtracts exactly that component. Everything perpendicular passes through untouched.

Anthropic released the lens but not the eraser, so I wrote the ablation myself: forward hooks with a second, clean copy of the model running one step ahead to supply the exemption lists. The subtle part is the control. Deleting ten directions damages any network, so at every single position and layer the control condition measures the exact size of the would-be deletion and removes an equally sized component along random directions at that same spot. Only the gap between the two arms counts as evidence about the workspace.

A tripwire instead of a leap

The plan, pre-registered before any run, was to replicate the chain-of-thought protection on GSM8K and then push it up a difficulty ladder (MATH-500, AIME) to see whether writing stops saving you when problems get hard. But the main grid costs about a day of compute, so I put a gate in front of it: the ablation first had to reproduce, on this model, the two signatures the paper found on frontier models.

Signature 1: deleting the workspace hurts reasoning far more than matched random damage.
Signature 2: ordinary next-token prediction on plain text survives.

Calibration was allowed to look at those two gauges and nothing else. In particular it never saw a chain-of-thought comparison, so I could not accidentally tune the intervention toward my own hypothesis.

What the tripwire caught

Signature 1 showed up beautifully. In the lightest configuration, deleting the top ten workspace directions cost the model 16 points on a two-step reasoning control while the exactly norm-matched random deletion cost zero (n=55 paired items, exact McNemar p=.012), with output staying fluent. The lens is pointing at real, causally load-bearing thoughts, even in a 4B model.

Signature 2 never showed up. At every strength I tried, workspace deletion also scrambled 34 to 42 percent of ordinary next-token predictions on plain text, roughly double the random arm’s damage, where the paper reports ordinary prediction left largely intact.

I also re-ran the entire ladder with a second lens I fit myself, targeting the paper’s preferred layer, to rule out an artifact of the off-the-shelf one. Its fit was small (32 prompts against the reference lens’s 479) and it produced something diagnostic: its “workspace directions” performed identically to the random control, drops of 10.9 versus 10.9. An undertrained lens converges to noise, which is exactly what noise should do, and it quietly confirms that the reference lens’s gap was real structure.

Eight configurations, two lenses, zero passes. Per the pre-registration, the main experiment never ran. Asking whether writing protects the workspace is meaningless on a model where trashing the workspace trashes everything else too.

One room, not two

Here is the picture I ended up with. In the frontier models of the paper, deliberate thought and automatic processing are two separate rooms: you can trash one and the other keeps humming. In Qwen3-4B they appear to be one room. The verbalizable directions exist and do real work, but everyday word prediction runs through the same machinery, so no deletion can hit one without the other.

Two readings survive, and they make opposite predictions. Either (a) the workspace/autopilot separation is something models grow into with scale, an emergent architecture of cognition, or (b) lens estimation at 4B is too blurry to isolate the workspace even though one exists. A supporting datum for (a): clean Qwen3-4B scores only 0.42 on the paper’s silent two-hop reasoning control (“the number of legs on the animal that spins webs”), where frontier models are near ceiling. The capability the workspace supposedly serves is mostly missing too.

The nice part is that the tiebreaker is cheap: public lenses exist for Qwen3-1.7B up through 32B. Run the identical gate up the size ladder and watch whether a second room appears. Reading (a) predicts the gate starts passing at some scale; reading (b) predicts it never does. That sweep is what I am doing next.

What I actually learned

The result I set out to get was never available, and the process is most of what I am taking away.

Negative results need more rigor, not less. A negative replication from an unknown student has an obvious dismissal: they fiddled until it broke. The only rebuttal is structure decided in advance: frozen thresholds, a control matched deletion-by-deletion, a calibration that provably never saw the hypothesis, and a written rule for when to stop. The stop-gate refusing to run my experiment is the finding, and it is only credible because refusing was specified before any data existed.

The deadliest bugs run clean and lie. Adversarial review of my own pipeline caught a control arm accidentally sorted by difficulty (which would have manufactured my exact hypothesis) and a bootstrap that deduplicated its own resamples, shrinking every confidence interval by a quarter. Neither ever threw an error. Both lived in which comparisons the data licenses, not in any line of code you could point to.

And the unglamorous 60 percent: macOS decided mid-experiment to prepare a system update and started silently killing my 8GB process every 90 seconds, faster than one generation could finish. The durable fix was not winning that fight but making death cheap: checkpoint every 64 tokens, replay the sampler state exactly on resume, restart forever while progress is being made. The experiment ultimately ran to completion on a laptop that was actively hostile to it.

This started as the entrance homework for Boaz Barak’s AI safety course at Harvard, grew into an abstract submitted to NEMI 2026, and the code, pre-registration, and full calibration records go public on August 6. The scaling sweep comes next. If the workspace really is emergent, I want to see the layer at which the second room gets built.