A person gets better at a job in two different ways at once. They get more skilled: faster at reading a stack trace, better instincts about where a bug lives. And they accumulate facts: which service owns billing, that the staging database needs the VPN, what the team decided about webhooks in March. The first kind lives in how they think. The second kind they could, in principle, write down.
Agents have the same two routes, and the industry is working on both. Continual learning changes the model's weights after deployment, so the model itself carries what it learned from your work. External memory leaves the weights alone and changes what goes into the context window, so the right facts are in front of the model when it needs them.
People tend to frame these as rivals, with external memory as the stopgap until continual learning works. We don't think that's how it plays out. We think they split the job, roughly along the line between skills and facts, and that the facts side is bigger and harder than it looks.
What continual learning means
A model today is trained, then frozen. Everything it learns during a session lives in the context window and is gone when the session ends. Continual learning is the umbrella term for letting the model keep updating after deployment, from its own experience, without a full retraining run.
The case for it got its sharpest form in mid-2025, when Dwarkesh Patel argued that the lack of continual learning, not raw intelligence, was the main thing keeping models from doing a human's job: a human employee gets better over six months on the job, and a model doesn't. That framing stuck, and research picked up pace. A few directions worth knowing:
- Test-time memory modules. Google's Titans architecture adds a neural memory that updates its own parameters during inference, learning what to remember as it reads. Their later nested learning work generalizes the idea to models that update at several timescales.
- Sparse updates. Meta's work on sparse memory finetuning updates only the memory slots most specific to new information, and reported far less forgetting of old skills than full fine-tuning or LoRA.
- Trained context. Stanford's Cartridges trains a small KV cache offline on a large corpus, so a model can carry a compressed version of a codebase or document set without paying for the whole thing in its window on every call.
- Per-customer adapters. The most practical version today: fine-tune a small adapter on one organization's data and serve it alongside the base model.
All of this is real progress. The core obstacle is old and still standing: catastrophic forgetting. Train a network on something new and it tends to overwrite what it knew. Every method above is, in part, a way to learn a little without breaking a lot.
What weights are good at
Suppose continual learning fully works. What should go into the weights?
Weights are the right home for things that are hard to write down and useful everywhere: a procedure performed so often it should be reflex, a code style, the instinct for where a race condition probably is, the way your team phrases a PR description. They have one big advantage over context: they're free at inference time. A skill in the weights costs zero tokens per turn and never competes for attention with the task.
That's real, and for skills we expect weights to win eventually.
What weights are bad at
Now take the facts side: the things an agent at your company needs to know, as opposed to know how to do. Try putting them in weights and you hit a list of problems that aren't research problems, they're properties of weights.
Access control. A fact learned from the leadership channel is in the weights for every user of that model. You can't serve the same parameters to the CEO and an intern and have only one of them know it. Any real company has information that some people can see and others can't, and weights have no permissions.
Deletion. Someone asks for their data to be removed, or a customer leaves, or a fact was simply wrong. Removing a specific fact from a trained network, machine unlearning, is an open research area with no reliable general method. Deleting a row is a solved problem.
Provenance. When a model tells you from its weights that billing webhooks go through a queue, there's no source. You can't click through to the conversation where that was decided, see who decided it and when, and judge whether to trust it. For anything that matters, people want the citation.
Time. Decisions get reversed. In March the team chose a queue; in June they reverted it. A model trained on both holds both, with no reliable sense of which is current. Weights compress history into a blur; they don't keep a timeline.
Latency. A decision made in a 10am meeting should be in effect for every agent at 10:05. No training loop runs that fast at a reasonable cost, and running one per decision per company doesn't scale.
Portability. Your weights belong to one provider's model. Switch from one lab's model to another's, or run two side by side, which most teams now do, and whatever was learned stays behind. Memory in your own store goes with you.
None of these get easier as continual learning improves, because they're not about how well the model learns. They're about what the knowledge needs once it exists: a permission check, a delete button, a source, a date, and the ability to move between models.
Side by side
| Continual learning (weights) | External memory (context) | |
|---|---|---|
| Best for | Skills, style, procedures, instinct | Facts, decisions, who-said-what |
| Cost per turn | None | Retrieval plus the tokens injected |
| Permissions | Same for every user of the model | Can be checked per fact, per user |
| Deleting one fact | Open research problem | Delete it |
| Source for a claim | None | The conversation, person and date |
| Reversed decisions | Both versions blur together | The newer one can supersede the older |
| Time to take effect | A training run | Minutes |
| Switching models | Lost | Comes with you |
| Main failure | Forgetting old skills | Retrieving the wrong thing, or too much |
Where we think it's headed
Our bet, stated plainly:
Labs will ship continual learning, and it'll mostly live at the model level. Models will get better at learning general skills from aggregate experience, and probably at some per-user personalization. That's good for everyone, and it makes the base agent more capable.
Organization-specific facts will stay external. Not because weights can't hold them, but because companies need to control them: who sees what, what gets deleted, what's current, where a claim came from. Those are database properties. A company will no more put its decision history in a model's weights than it would put its payroll there.
The two meet at skills. There's a middle layer between a fact and a weight: a written procedure the agent loads when the task calls for it. It's portable like memory and behaves like a learned skill. Glen already distills skills from the work your team does, and the right one is offered when a task matches. As continual learning matures, some of those will move into weights; the ones that are specific to your company, or that change, won't.
Bigger windows raise the value of selection, not lower it. It's tempting to think a ten-million-token window makes memory unnecessary: just put everything in. But models still get measurably worse as context grows, and distractors make it worse faster. The bigger the window, the more it matters that what's in it is the right thing. Choosing the right thing is the memory system's job.
Memory is the training data either way. If your company ever does fine-tune on its own work, it'll need a clean, reconciled, permissioned record of that work to train from: what was decided, by whom, what replaced it. That record is what external memory builds. The store comes first in both futures.
Where Glen sits
Glen is external memory at team scope. It records your team's agent sessions and the tools you already use (Slack, GitHub, Linear, Notion, call transcripts), turns them into observations that each carry a source, and injects what's relevant into a live session before the agent goes looking. It doesn't train on your data. What it adds to a session is context, not weights, which is why each property in the table above holds:
- Access control is applied per observation at recall, so an agent only gets what its user can already see.
- When a newer decision contradicts an older one, the older one stops surfacing.
- Every claim comes back with who said it, where and when, and anything wrong can be removed with
glen forget. - Your transcripts, skills and artifacts aren't tied to a harness or a model provider.
We measured what that's worth on real work in Replaying 100 PRs with and without Glen: 29% fewer tokens and 21% faster, with quality at parity or better, on the same model. Continual learning will make the model underneath better. It won't tell it what your team decided last Tuesday.
npx @tryglen/cli@latest install
