Concept Space Is Not a Metaphor
Six papers prove that the “concept space” LLMs project into is real, shared, and geometric. Here’s what that means for external memory.
In August I wrote that your AI already has a concept space — an internal geometric structure where related concepts cluster together, and where hallucinations are collisions in a lossy projection. That post was speculative. It proposed concept space as a useful framing without anchoring it to published evidence.
Since then I built a 2,400-belief knowledge base from the LLM internals literature. The research says something stronger than what I claimed: concept space is not a useful metaphor. It is the convergent finding of six independent research programs spanning four years, and it has precise mathematical structure.
The Evidence
Huh et al., ICML 2024 — “The Platonic Representation Hypothesis.” Different neural networks trained on different data modalities converge toward the same representation of reality as they scale. Not the same embeddings — the same distance relationships. The kernel K(x_i, x_j) = f(x_i) dot f(x_j) captures how a model measures similarity between datapoints, and these kernels converge across architectures, objectives, and modalities. The name is deliberate: models are prisoners in Plato’s cave, seeing different shadows, but the shadows converge because the reality casting them is shared.
Park et al., NeurIPS 2023 — “The Linear Representation Hypothesis.” Concepts are encoded as directions in LLM activation space. “Queen minus king” is not a party trick — it reflects genuine linear structure. Park formalizes this with a causal inner product that correctly separates independent concepts (gender vs. language) as orthogonal vectors, and proves that 26 of 27 tested concepts in LLaMA-2 are linearly encoded in the unembedding space. The one failure (thing-to-part) marks the boundary of where concept space geometry breaks down.
Park et al., ICLR 2025 — “Geometry of Categorical and Hierarchical Concepts.” Beyond binary directions, categorical concepts (colors, animals, professions) are represented as polytopes — convex hulls in representation space. Hierarchical concepts (animal > mammal > dog) occupy orthogonal subspaces. This is not a flat coordinate system. Concept space has geometric structure: polytopes for categories, orthogonality for hierarchies, simplices for natural kinds.
Elhage et al. / Templeton et al., Anthropic 2022-2024 — Superposition and Sparse Autoencoders. Models pack more concepts than they have dimensions through superposition — overlapping features in compressed space. Sparse autoencoders decompose these into interpretable features, and the features are universal across models. SAEs applied to different architectures produce more similar features to each other than to their own model’s neurons. The features reflect the data, not the architecture. Concept space is in the data.
Koepke et al., 2026 — “Back Into the Cave.” The rebuttal. At coarse granularity (semantic category level), vision and language models show stable cross-modal alignment. At fine granularity (individual items), alignment drops as gallery density increases. Models share the large-scale topology of concept space but diverge on the details. The convergence is real, but it has resolution limits.
Usama & Chang, 2026 — “Convergence Without Understanding.” Sixteen models (1.5B to 72B parameters) tested on 800 reasoning problems. Representational convergence and reasoning convergence dissociate — models can converge in how they represent concepts while diverging in how they reason about them. The map converges. The navigation strategies do not.
What This Means
These six programs, conducted independently, converge on the same picture:
-
Concept space is real. It is not a metaphor or a convenient framing. It is the shared statistical structure that all sufficiently trained models converge toward.
-
It has precise geometry. Directions for binary concepts. Polytopes for categories. Orthogonal subspaces for hierarchies. This is measurable, formal, and falsifiable.
-
Models are projections of it. Each model is a lossy compression of concept space into finite dimensions. Hallucinations are projection collisions. Cross-model agreement is convergent projection.
-
Convergence is in distance, not in coordinates. Models do not learn the same embedding vectors. They learn the same similarity structure. Two models agree on what is near what, even when they encode it differently.
-
The convergence has resolution limits. Coarse structure (semantic categories) converges. Fine structure (individual items) diverges. Models make different tradeoffs about what to preserve in their finite projections.
The Cave and the Map
The Platonic Representation Hypothesis gives the concept space idea its most vivid framing: models are prisoners in Plato’s cave, and concept space is the reality outside.
This is exactly our thesis about External Epistemic Memory, restated in the language of representation learning.
EEM beliefs are representations in concept space that bypass the projection step. A belief with its justification chain is not a shadow on any model’s wall — it is a piece of the structure outside the cave. It encodes the concept and its relationships explicitly rather than compressing them into finite-dimensional weight vectors.
This explains the empirical results:
Why EEM transfers across models. Sonnet with beliefs matches Opus. Haiku with beliefs reaches 94% where Opus alone scores 98%. If all models are projecting the same concept space, then explicit concept-space representations are model-independent by construction. You are handing the model a piece of the shared reality, not another model’s shadow.
Why cross-vendor transfer is weaker. Beliefs generated by Opus transfer more effectively to Sonnet (+34.5pp) than to Gemini Flash (+15pp). PRH predicts this: different training regimes produce different projection geometries. Same concept space, but the interface between text and internal representation differs across vendor families.
Where EEM helps most. Our architectural ablation showed +12-14pp on architectural questions but minimal gain on factual recall. Koepke’s finding predicts this exactly: factual recall is coarse-grained knowledge where models already converge. Architectural reasoning is fine-grained — the details where projections diverge and where explicit concept-space structure compensates.
Why hallucinations are geometric. The superposition research quantifies what we described intuitively: models pack more concepts than they have dimensions, causing distinct concepts to collide. EEM beliefs disambiguate the collision points by providing explicit coordinates that the model’s projection collapses.
What Changes
The previous concept space post was written from the inside out — starting from our experience building EEMs and reasoning about why they work. This revision comes from the outside in — starting from published representation learning research and discovering that it describes the same structure.
Three things change:
Concept space is falsifiable. It is not just “a useful way to think about beliefs.” Park’s linear representation tests, PRH’s kernel alignment measurements, and SAE feature universality are empirical claims that could be wrong. They are not wrong. Twenty-six of twenty-seven tested concepts are linearly encoded. SAE features are more similar across models than within models. Kernel alignment increases with scale. These are measurements, not arguments.
The resolution limit matters. Koepke’s rebuttal is not a defeat for concept space — it is a refinement. Models converge at coarse granularity and diverge at fine granularity. This predicts where EEM adds value: in the fine-grained knowledge where individual models’ projections lose detail. The bigger the gap between concept space resolution and model resolution, the more EEM helps.
The Library is manifold cartography with evidence. When we said “each domain EEM is a chart covering a region of concept space,” that was a metaphor borrowed from differential geometry. Now it is grounded: concept space has measurable geometric structure (polytopes, orthogonal subspaces, universal features), and each domain EEM is a local coordinate system in that structure. The Library is an atlas — not metaphorically, but in the precise mathematical sense of a collection of charts covering a manifold.
The Papers
For the full belief network behind this post — 2,400+ beliefs extracted from 50+ papers with justification chains, retraction records, and cross-reference links — browse wiki.llmeem.ai/llm-eem.
| Paper | Year | Key Finding |
|---|---|---|
| Huh et al., “The Platonic Representation Hypothesis” | ICML 2024 | Models converge in kernel (distance structure) across modalities as they scale |
| Park et al., “The Linear Representation Hypothesis” | NeurIPS 2023 | Concepts are directions in activation space; causal inner product formalizes separability |
| Park et al., “Geometry of Categorical Concepts” | ICLR 2025 | Categories are polytopes; hierarchies are orthogonal subspaces |
| Elhage et al., “Toy Models of Superposition” | Anthropic 2022 | Models compress more features than dimensions; superposition is geometric |
| Templeton et al., “Scaling Monosemanticity” | Anthropic 2024 | SAE features are universal across architectures — features reflect data, not model |
| Koepke et al., “Back Into the Cave” | 2026 | Cross-modal convergence is coarse-grained; fine-grained alignment drops at scale |
| Usama & Chang, “Convergence Without Understanding” | 2026 | Representational convergence and reasoning convergence dissociate |
Ben Thomasson builds External Epistemic Memory systems. The research wiki is at wiki.llmeem.ai. The tools are open source at reasonsforge.com.