Reasoning in the Dark: The Coming Trade Between Efficient Thought and Auditable Thought

Can a model think in a language you cannot read?

That question sounds abstract until you watch it happen. A few weeks ago I was reviewing a reasoning trace from a model that had been fine-tuned to compress its chain of thought. The visible output was clean: a short, confident answer with a tidy three-line justification. But the internal logs told a different story. The model had run something closer to a breadth-first search over half a dozen competing hypotheses, collapsed them into a single latent representation, and only then emitted the human-readable summary. The summary was not the reasoning. It was a press release written after the reasoning had already happened somewhere I could not see.

In my experience building AI systems at scale, that gap between what a model shows you and what it actually did is the thing that quietly breaks trust. And right now, the entire field is racing to widen it. Not maliciously. For a very good reason: reasoning in natural language is slow and expensive, and reasoning in continuous latent space is fast and cheap. The problem is that the token trace we are throwing away is also the only real window we have ever had into how these systems think.

This is the trade I want to sit with in this article. We are moving from models that reason out loud to models that reason in the dark, and we are calling it progress. Some of it genuinely is. Some of it is going to cost us more than we currently admit.

The Shift Is Real, Not Hype

Let me start by taking the optimists seriously, because the architectural shift here is substantive. This is not a rebranding of prompt engineering. It is a coherent research program with a shared premise: natural language may be a suboptimal medium for reasoning.

The canonical example is Meta FAIR's Coconut, short for Chain of Continuous Thought. Instead of decoding each intermediate reasoning step into words and feeding those words back in, Coconut takes the model's last hidden state and feeds it directly back as the next input embedding. The reasoning never leaves continuous space. The paper describes it plainly: the method uses the last hidden state as a representation of the reasoning state, and instead of decoding it into words, feeds it back to the model as the next input embedding directly in the continuous space.

Why does that matter? Because a token is a commitment. When a model writes the word "therefore," it has collapsed a distribution over possible next thoughts into a single discrete choice. A continuous thought does not have to commit. The Coconut authors point out that a continuous thought can encode multiple alternative next steps at once, letting the model perform something like a breadth-first search rather than locking prematurely onto one deterministic path. On logical reasoning tasks that require real search during planning, Coconut outperforms chain-of-thought while using fewer thinking tokens at inference.

Coconut is not alone. It sits inside a family of architectures all pushing reasoning off of discrete tokens and into richer latent units:

Architecture	The unit of "thought"	Core claim
Coconut (Chain of Continuous Thought)	Fed-back hidden state	Reasoning stays in continuous space; encodes multiple paths at once
Large Concept Models (LCM)	Sentence embedding (a "concept")	Predict the next concept, not the next token
Dynamic LCM	Learned semantic boundary	Discover where concepts begin and end, then compress
Byte Latent Transformer (BLT)	Entropy-based byte patch	Tokenizer-free; allocate compute where data is complex
Recurrent-Depth (Huginn)	Iterations of a recurrent block	Scale test-time compute by unrolling depth, not emitting tokens

The Large Concept Model is the most conceptually aggressive of these. It moves the atomic unit of prediction from the token all the way up to the sentence, represented as an embedding in a space called SONAR, and trains the model to autoregressively predict the next concept. The motivation, as the LCM literature frames it, is to reason over higher-level semantic entities as atomic units, in contrast with traditional models that operate at the lexical token level. Dynamic LCM then removes the assumption that concepts conveniently align with sentence boundaries, learning where the boundaries actually fall and shifting computation into a compressed concept space where reasoning is cheaper.

Then there is the Byte Latent Transformer, which I find the most quietly important of the group. BLT is tokenizer-free. It groups raw bytes into dynamically sized patches based on next-byte entropy, spending more compute where the data is genuinely complex and less where it is predictable. The headline result is the one that should make anyone paying attention sit up: at fixed inference cost, BLT shows significantly better scaling than tokenization-based models. That is not an efficiency footnote. That is a claim that the token, the fundamental unit we have built every large language model around, may itself be the bottleneck.

The Economics Problem Latent Reasoning Is Trying to Solve

To understand why anyone would abandon a perfectly readable reasoning trace, you have to understand what the industry has been doing to squeeze more reasoning out of existing models. The dominant strategy of the last two years has a name: test-time scaling. Spend more compute at inference, and the model reasons better.

It works. It also does not scale the way the marketing implies. "Test-time scaling" is not one technique, it is a family of them: extending deliberation along a single trajectory, sampling many candidate answers and voting, searching over partial solution states. They differ in cost and, importantly, in failure modes. And when you look at the actual return on compute at the frontier, the picture gets uncomfortable.

Here is the number that reframed the whole thing for me. In one 2026 analysis, inference scaling improved accuracy by up to 7.1 points over chain-of-thought, but only at the highest evaluated budget, which was roughly 20 times the compute of ordinary chain-of-thought, on MMLU-Pro. Twenty times the compute for seven points. That is not a curve you extrapolate happily. An infrastructure-level study of dynamic reasoning put it bluntly: agents improve accuracy with more compute but suffer from rapidly diminishing returns, widening latency variance, and unsustainable infrastructure costs.

So the economic logic of latent reasoning becomes obvious. If most of your inference cost is spent emitting and re-ingesting reasoning tokens, and those tokens are just a lossy serialization of a computation happening inside the model anyway, why pay for the serialization? Keep the reasoning in latent space. Iterate a recurrent block to arbitrary depth, as Huginn does, and scale test-time compute without producing a single extra visible token. The recurrent-depth work makes this explicit, contrasting itself with mainstream reasoning models that scale compute by producing more tokens.

I want to be fair here: this is a legitimately elegant answer to a real cost problem. There is even evidence that test-time compute, spent well, beats raw model scaling on the accuracy-per-unit-energy trade-off for genuinely reasoning-heavy tasks. If your only concern were the compute bill, latent reasoning would be an easy call.

But the compute bill is not the only concern. And this is where I part ways with the consensus.

The Twist: What We Lose When Reasoning Goes Dark

Here is the uncomfortable part. The chain-of-thought trace we are optimizing away is not just a computational artifact. For the last two years it has quietly been one of the most important safety tools we have, and almost nobody was treating it as one.

Think about what an explicit reasoning trace gives you. When a model reasons in readable text, you get a window, imperfect but real, into its cognition. You can inspect it for deceptive intent, for misaligned goals, for the moment a model decides to pursue an objective you did not sanction. The safety literature is explicit that generating intermediate reasoning steps provides a human-readable window into a model's cognition, enabling monitoring for deceptive or misaligned behavior.

Latent reasoning closes that window. And the subtle point, the one I think is most often missed, is what kind of thing the latent trace actually is. A recent causal-geometric analysis of latent reasoning models put it in a way that has stuck with me: latent thoughts should be treated as hidden computation, not hidden explanation. The fact that you can sometimes decode a latent vector into something that looks like a thought proves nothing about whether that decoded thought is actually driving the model's behavior. Decodability is not mechanism.

That distinction matters enormously. With chain-of-thought, at least in principle, the text the model produces is on the causal path to its answer, so reading it tells you something. With latent reasoning, you might decode a plausible-looking explanation from the hidden state that has nothing to do with the computation that produced the output. You would be reading tea leaves and calling it interpretability.

Now, the honest counterargument, and I want to give it real weight, is that chain-of-thought was never as trustworthy as its defenders claim. This is true. Models do not always faithfully represent their internal decision-making in the reasoning they generate. There is a whole literature on chain-of-thought unfaithfulness showing that the stated reasoning and the actual computation can diverge. So one could argue we are giving up a window that was already smudged and cracked.

I hear that, but I do not find it reassuring, and here is why. A smudged window is still a window. The research on when chain-of-thought is necessary found that when a task genuinely requires serial reasoning, models struggle to evade a monitor watching their chain of thought, which means the monitoring offers a substantial, if imperfect, layer of defense that is worth actively protecting. A cracked window you can still occasionally see through beats a wall. Latent reasoning does not clean the window. It bricks it up.

There is a sliver of hope in the details. Early work on detecting misaligned reasoning in continuous-thought models suggests that misalignment tends to concentrate in the early "planning" latent tokens, which hints that monitoring could target that phase specifically. That is a real research direction and I am glad people are pursuing it. But it is a long way from a deployable safety guarantee, and we are shipping latent-reasoning architectures now, not after that research matures.

What This Means If You Are Building With These Models

It would be easy to read this as a case against latent reasoning. It is not. The efficiency gains are real, the research is good, and the direction of travel is not going to reverse because a blog post raised concerns. The useful question is not whether to adopt these architectures but how to adopt them without walking blind into the interpretability gap.

A few things I have come to believe matter.

Treat the reasoning trace as a feature, not a byproduct

When you evaluate a reasoning model for production, do not just measure accuracy and latency. Measure how much of the model's actual decision process is inspectable. If you are choosing between a latent-reasoning model that is two points more accurate and a chain-of-thought model whose reasoning you can audit, the audit-ability has a dollar value, and in regulated domains it may dominate the accuracy difference. Most current evaluation harnesses do not measure this at all. That is a gap worth closing before you need it, not after.

Match the architecture to the stakes

Latent reasoning is a great fit for tasks where the output is cheaply verifiable and the cost of a wrong answer is low: search-heavy planning, code that either passes tests or does not, tasks with a hard external check. It is a much worse fit for high-stakes, low-verifiability decisions where you would want to understand the reasoning precisely because you cannot easily check the answer. The mistake I expect to see repeatedly over the next 18 months is teams adopting latent reasoning uniformly, for the efficiency, and then discovering they have deployed it in exactly the places where auditability mattered most.

Invest in mechanistic checks, not decoded narratives

If you are going to run latent-reasoning models in production, the interpretability work you fund should be causal and mechanistic, not the comforting kind that decodes a hidden state into a nice-sounding explanation and calls it a day. Ask whether an intervention on the latent state actually changes the output in the way the decoded narrative predicts. If it does not, you do not have an explanation. You have a story.

None of this is a reason to sit out the shift. It is a reason to go into it with your eyes open, which is harder than it sounds when the efficiency numbers are this seductive.

Where This Leaves Us

The arc of this research is one of the more consequential things happening in AI architecture right now, and it is happening quietly, under headlines dominated by model releases and benchmark records. Reasoning is migrating off of human-readable tokens because tokens are expensive and, it turns out, may not even be the best substrate for thought. Coconut, Large Concept Models, the Byte Latent Transformer, recurrent-depth models: these are not incremental. They are a bet that the next generation of reasoning happens in a space we cannot natively read.

I think that bet is going to pay off on efficiency. I am much less sure it is going to pay off on trust. The last two years gave us a rare gift in AI safety, which is that our most capable reasoning models happened to think in a language we could read. That was never guaranteed, and it appears to have been temporary. We are trading it away for speed.

Maybe that trade is worth it. Maybe the interpretability research catches up and we learn to monitor latent reasoning as well as we ever monitored chain-of-thought. But I would rather the field make that trade deliberately, with a clear-eyed accounting of what is being given up, than sleepwalk into it because the compute savings were too tempting to question. Efficient thought and auditable thought are both worth having. The uncomfortable truth of this moment is that, for now, we are being asked to choose.

Key Takeaways