The Context Window Is a LoRA You Already Have
How structured beliefs steer LLM activations without touching weights — and why this changes the economics of AI reasoning.
Every organization using LLMs wants the same thing: make the model an expert in their domain. The standard approach is fine-tuning — LoRA adapters that modify model weights to steer behavior. This works, but it requires access to weights, training infrastructure, GPU time, and a team who knows how to build datasets. Most teams do not have these things.
There is another way. One you already have access to. It requires no weights, no training runs, and no GPUs. It works across models, is human-readable, and can be updated in minutes instead of days.
The context window is a LoRA adapter. You just need the right content to put in it.
How LoRA Works
LoRA (Low-Rank Adaptation) injects small weight matrices into a frozen model that alter how information flows through the transformer layers. These matrices change permanent pathways — the model’s behavior shifts for every input, because the weights themselves are different.
The context window does the same thing through a different mechanism. When you fill the context window with text, Multi-Head Attention calculates similarity between the input query and the context content. Where similarity is high, Value vectors from the context are mixed into the hidden states, physically steering downstream activations toward a different region of the model’s internal representation.
LoRA changes permanent pathways via altered weight matrices. The context window changes real-time signal routing via attention. Different mechanisms, same behavioral shift.
The Tradeoff That Disappears
The two approaches have complementary weaknesses:
| Context Window | LoRA | |
|---|---|---|
| Permanence | Temporary — gone when the session ends | Permanent — saved as a reusable file |
| Per-query cost | High — the full context is processed every prompt | Low — normal speed, no token overhead |
| Training cost | Zero — just fill the context | High — needs dataset, GPUs, expertise |
| Weight access | Not required — works with any API | Required — needs model internals |
| Updateable | Instantly — change the text | Expensive — retrain the adapter |
Each one is strong where the other is weak. What if you could combine the strengths of both?
External Epistemic Memory dissolves the tradeoff:
- Permanence: beliefs persist in an external database. They survive session boundaries, context compaction, model swaps. The “forgotten when chat ends” problem disappears.
- Per-query cost: dynamic retrieval (
deep_search,explain_belief) loads only the relevant slice, not the entire knowledge base. Cost scales with query relevance, not total knowledge size. - Training cost: zero. Beliefs are built by the LLM itself through the derive-then-review pipeline. No datasets, no GPUs, no ML engineering.
- Weight access: not required. Works with any API-accessible model.
- Updateable: retract a belief and its dependents cascade out automatically. Try doing that with a LoRA adapter.
Why the LLM Must Build It
You cannot manually write effective context-window LoRA because you do not know the model’s internal representations. You would be guessing at which token sequences activate which concepts — writing prompts for a system whose wiring diagram you have never seen.
But an LLM knows, implicitly, what token sequences activate which internal states. When an LLM processes source material and produces structured beliefs, it is translating from human-readable text into a form that optimally activates models sharing its training distribution. The derive step is the LLM building its own LoRA in token space.
This is why prompt engineering is hard and often feels like guesswork. The user is trying to hit a target they cannot see, in a space they cannot map. The LLM can hit the target because it lives in that space. Let it write the adapter.
Semantic Resonance
Here is the surprising part: LLM-generated beliefs activate the model more precisely than human-written text with identical semantic content.
When an LLM generates a belief, the output text is synthesized according to the model’s own statistical weights. The vocabulary, syntax, and structure form a precise key that triggers nearly identical projections back into the target activation states. Human-written text includes stylistic noise, ambiguous phrasing, and vocabulary choices that scatter attention across concept space instead of snapping to the target.
We call this semantic resonance — the output of one inference pass becomes a high-fidelity input for the next. The model “re-experiences” its previous reasoning rather than cold-starting.
This creates a self-consistency loop: the model generates beliefs optimized for its own internal representations. Those beliefs persist externally. When loaded into a future session, they reactivate the same conceptual pathways with minimal signal loss. Prior cognition is not replayed — it is re-triggered.
And because models trained on similar data develop similar internal geometry, beliefs generated by one model activate similar regions in other models. We measured this: Sonnet with beliefs built by Opus matches Opus without beliefs. The coordinates transfer because the underlying concept space is shared.
Dynamic Assembly
A static LoRA is fixed — built ahead of time, applied uniformly to every query. If the query does not match what the LoRA was built for, the adapter is wasted context or actively harmful.
EEM is a dynamic LoRA. The model queries the knowledge base with tools like deep_search and explain_belief. Each query returns pre-computed, reviewed, justified beliefs tailored to exactly what the model needs for this specific problem. The model assembles its own adapter from an arbitrarily large knowledge base, loading only the relevant slice.
This is the model as its own adapter engineer. It reads the problem, formulates queries, retrieves the relevant beliefs, and conditions its own generation on them. No human needs to decide which adapter to load. No routing logic selects between pre-built LoRAs. The model is selecting, composing, and applying semantic weights in real time.
The knowledge base behind the dynamic LoRA can be arbitrarily large — 12,000+ beliefs in our largest domain, scaling to millions across federated domains. The context window is not the bottleneck. It is a narrow, high-bandwidth channel into a knowledge base that would never fit in context all at once.
Persistent Thinking Tokens
Extended thinking and chain-of-thought reasoning improve inference quality by inflating token count — spend more compute, get better answers. But the thinking is ephemeral. Every session pays the full cost again.
EEM beliefs are the inverse: persistent thinking tokens that decrease token count.
| Approach | Token Direction | Persistence | Cost Trajectory |
|---|---|---|---|
| Extended thinking | Inflates tokens | Ephemeral | Increases each session |
| Chain-of-thought | Inflates tokens | Ephemeral | Increases each session |
| EEM beliefs | Decreases tokens | Persistent | Decreases each session |
A belief is a minimum retrieval key for continued cognition — the smallest possible text that reactivates a specific region of concept space. A 50-100 token belief achieves the same cognitive activation as a 5,000 token raw file. Same activation, 50-100x fewer tokens.
This inverts the economics of AI reasoning. In the current model, better reasoning means more tokens means higher cost. Quality and cost move together. With EEM, better reasoning means fewer tokens means lower cost. Quality and cost move in opposite directions.
The derive-then-review pipeline pays the thinking cost once. Every subsequent session gets the thinking for free.
The Numbers
We measured this across two domain builds:
| Domain | Items | Build Cost | Derive Cost | Derive % |
|---|---|---|---|---|
| 800 Jira issues | 5,460 beliefs | $179 | $6.32 | 3.5% |
| 625 code files | 1,396 beliefs | $136 | $5.85 | 4.3% |
Derivation — where the system generates new insights by reasoning over compressed beliefs — costs less than 5% of the total build. Reading raw material is expensive. Thinking over pre-compressed beliefs is almost free. The model accesses the most valuable parts of its representation at minimal token cost because beliefs are already in its native semantic space.
And the build cost is one-time for the entire organization. A team of 10 engineers each re-reading 30 code files per session burns $990/month on redundant reading. The EEM costs $136 once, then $20/month maintenance. Annual savings: over $11,000 — and that is before counting the inference quality improvement.
What This Means
You do not need to fine-tune your model. You do not need weight access, training infrastructure, or an ML team. You need structured knowledge in the right format, built by the model itself, persisted externally, and loaded dynamically at query time.
The context window is the adapter mechanism. The beliefs are the weights. The LLM is the engineer that builds them.
The tools to do this exist now, are open source, and work with any model you have API access to.
Ben Thomasson builds External Epistemic Memory systems. The tools are open source at reasonsforge.com. The research is at llmeem.ai.
Comments (4)
Really sharp article, Ben. The framing of the context window as a dynamic, real-time activation router—rather than just a passive buffer—is a fantastic mental model.
A few thoughts on where this hits home and one nuance worth drilling into:
Overall, this is a compelling argument for moving away from heavy fine-tuning pipelines when dynamic, structured context management can achieve similar activation control without the training overhead. Looking forward to keeping an eye on the reasonsforge repo!
This is a compelling framing of EEM, and I think the strongest idea here is actually more interesting than the “context window = LoRA” analogy.
The key distinction, to me, is that LoRA performs parameter-space adaptation, while EEM performs persistent, externally stored, dynamically retrieved activation conditioning. The mechanisms are different, but the functional objective is similar: cause a frozen model to behave as though it has acquired something it did not previously have. That makes “dynamic LoRA” a useful analogy, provided it is understood as an analogy rather than a literal equivalence.
The section on semantic resonance is where I think the really interesting research claim begins. The idea that an LLM-derived belief is a particularly efficient representation for reactivating the model’s own conceptual machinery is much stronger than ordinary RAG or summarization. If the claim holds experimentally, it means the derive step isn’t merely compressing information—it is translating information into a representation that is unusually compatible with subsequent inference.
I’d be especially interested in seeing the methodology behind the Opus → Sonnet result. “Sonnet with beliefs built by Opus matches Opus without beliefs” is potentially a very significant result, but “matches” needs a precise operational definition. Accuracy, answer agreement, activation similarity, reasoning quality, token efficiency, or some combination would lead to very different interpretations. Likewise, the claim that a 50–100 token belief produces the “same cognitive activation” as a 5,000-token source file sounds like exactly the sort of thing that could turn this from a compelling hypothesis into a measurable research result.
I also think the persistent thinking tokens concept is excellent. The important economic insight isn’t really that thinking becomes free—it is that reasoning becomes amortized. Instead of paying repeatedly to rediscover the same useful conclusions, the system pays the derivation cost once and subsequently retrieves a compact representation of that work. That’s a potentially important distinction from conventional RAG, where the system repeatedly pays to process source material.
The other comparison I’d like to see made explicit is EEM versus RAG. RAG retrieves documents; EEM appears to retrieve derived epistemic state—beliefs, dependencies, justifications, and potentially retractions. That distinction seems fundamental. If the system can preserve not just what was concluded but why, and can propagate belief retractions through dependents, then EEM is doing something qualitatively different from simply putting better chunks into a context window.
The one place I’d be cautious is the language around weights and LoRA. “The beliefs are the weights” is a great slogan, but technically the beliefs aren’t weights; they are inputs that cause the unchanged weights to produce different activation trajectories. I think the article becomes stronger, not weaker, if it makes that distinction explicit. The interesting claim is precisely that you can get an adapter-like effect without an adapter.
Overall, I think the most important thesis here is:
EEM turns accumulated reasoning into a persistent, dynamically assembled, token-space adaptation layer.
If the semantic-resonance and cross-model-transfer results hold up under controlled experiments, that is a considerably more interesting result than the LoRA analogy itself.
Solid post. The core move — treating structured, LLM-authored beliefs in an external store as a dynamic, retrievable substitute for a LoRA — is the right framing for a lot of real organizational work. Most teams cannot fine-tune; almost all of them can put text in context. Making that explicit and then closing the permanence / cost / updateability gaps with a proper belief graph is useful.
A few reactions, in order of how much they moved me:
The economics section lands hardest. “Persistent thinking tokens that decrease token count” is a clean inversion of the usual CoT / extended-thinking story. Paying the expensive pass once (derive + review) and then re-activating with 50–100 token keys is exactly the amortization people want. The Jira and code-file numbers are directional but credible; the “team of 10 re-reading the same files” example will resonate with anyone who has watched inference bills.
Semantic resonance is the claim that needs the most evidence. The intuition is good: the model’s own phrasing is a better key into its own geometry than a human paraphrase of the same content. The cross-model transfer anecdote (Sonnet + Opus-built beliefs ≈ Opus alone) is suggestive. I’d love to see the actual measurement — representation similarity, probe accuracy, or even just controlled behavioral tests with human-written vs. model-written beliefs of matched semantics. Without that, it stays a strong hypothesis.
The LoRA analogy is rhetorically effective and mechanistically a bit loose. Attention does steer activations by mixing values; LoRA changes the weights that produce those activations. Same behavioral outcome is plausible; “same thing through a different mechanism” is stronger language than the evidence currently supports. I’d keep the analogy as a teaching device and be more careful about equivalence claims.
Dynamic assembly is the practical win. Static adapters are brittle; a retrieval interface over a reviewed belief store lets the model compose the conditioning set per query. That is real, and it is what makes the “arbitrarily large knowledge base, narrow high-bandwidth channel” picture work. The cascading retract is the part that actually distinguishes this from garden-variety RAG + memory — try doing dependency-directed backtracking with a LoRA file.
One practical note: the footer points at reasonsforge.com / llmeem.ai. The implementation is also on PyPI as
reasonsforge(Reason Maintenance System, Doyle-style nodes/justifications/nogoods, forge pipelines for code/product/project, derive/retract/explain, etc.). Making that install path obvious would help the “tools exist now, are open source” closer land for people who want to try it the same day they read the post.Net: strong conceptual and economic argument, useful for practitioners who are tired of fine-tuning friction. The mechanism and measurement claims are the main places I’d tighten before treating this as research rather than advocacy. Looking forward to the follow-ups on concept space and the library.
This framing earns its title — the analogy is more than rhetorical, and there’s actual literature backing the connection you might want to cite: Dai et al. (“Why Can GPT Learn In-Context?”) and von Oswald et al. both show in-context learning performing something like implicit gradient descent / weight updates under simplifying assumptions. Your post is essentially the systems-engineering consequence of that theory, and I don’t think I’ve seen anyone else draw the practical conclusion this cleanly: if context is an adapter, then the interesting engineering problem is adapter construction, and the derive-then-review pipeline is a training loop that runs at inference prices.
That said, a few pushbacks. First, the equivalence is looser than the “How LoRA Works” section implies. LoRA modifies computation at every layer for every token, including the ones being generated; context steering only acts through attention over the context’s KV cache. These overlap a lot behaviorally but aren’t interchangeable — LoRA can globally suppress a behavior or shift style in ways that are hard to achieve by injection, and conversely context can carry fresh facts that low-rank finetuning absorbs poorly. Honestly the nearer neighbor to your mechanism is prefix/soft-prompt tuning, which literally optimizes virtual context tokens. “The context window is a prefix-tuning adapter” is a worse title but a tighter claim.
Second, “semantic resonance” is doing a lot of load-bearing work while being the least evidenced section. The claim that model-generated beliefs activate the model more precisely than human-written text with identical semantic content is testable, and I’d love to see the ablation: same facts, human phrasing vs. model phrasing, measure downstream accuracy. Until then, the boring alternative explanation — model-generated beliefs are simply more uniform, better structured, and less ambiguous than human notes — accounts for the same observations without any appeal to internal geometry.
Third, you buried your best result. “Sonnet with beliefs built by Opus matches Opus without beliefs” is a headline finding — capability distillation through context, with real cost implications — and it appears as a single sentence mid-post. What benchmark, what n, what variance? If that holds up under scrutiny it deserves its own post more than the cost tables do.
On the numbers: the $11K savings calculation counts raw-file re-reading as the counterfactual, but a fair comparison should include prompt caching (which slashes the cost of redundant reading), the retrieval tool-call overhead of deep_search itself, and the degenerate competitor — a hand-maintained CLAUDE.md, which is basically a one-belief EEM and is what most teams do today. EEM probably still wins at 12K beliefs where a flat file can’t scale, but that’s the argument to make explicitly.
None of this undercuts the thesis, which I think is right and underappreciated: adaptation via retrieval-assembled context gets you most of what finetuning promises with none of the infrastructure, and it stays updatable. The tradeoff table alone is worth bookmarking. Mostly I want the resonance claims to catch up to the confidence of the prose — the framework is strong enough that it doesn’t need them stated as fact yet.