Back to the blog

AI Governance

Your AI Will Soon Learn on the Job. Is Your Governance Ready?

By John Utley

Published

An operator maps the connected stores used by several AI agents.

Most of the AI in a mid-market company is still frozen at launch. It knows its training data, plus whatever fits in the current chat. Then the session ends, and the lesson disappears. Teams try to fix that in the wrong places: an unreviewed prompt tweak, a retrieval store nobody versions, a fine-tune with no rollback.

Research is moving the other way. Papers on continual learning, including Google's Nested Learning work, argue that a model can be a stack of memories that update at different speeds. That research is not a product you can buy or deploy today. Hope, the proof-of-concept model in that paper, is not something Sophizo resells.

Sophizo does not sell Nested Learning or the Hope model.

The useful idea for operators is simpler: if AI will keep learning after launch, governance built for a one-time validation will age poorly.

What breaks first is the validate-once habit. A risk review that signs off a frozen model does not automatically cover a memory that changes every week. If you cannot show which version of the model and the memory produced a decision, you cannot investigate it. If no one is named as the owner of what the system is allowed to remember, lessons become policy in the dark.

You do not need a new model to close that gap. Design memory in layers. Fast memory is this task. Medium memory is the daily record. Slow memory is approved policy, playbooks, and definitions. A lesson moves down a layer only after a person reviews it, with an owner and a log.

Version the model and the memory. Write down what would trigger a re-validation after launch. Map those controls to NIST AI RMF Measure and Manage, and to ISO 42001 monitoring and continual improvement.

Sophizo already uses this pattern in ForecastIQ, which stays included in Agentic RevOps: weekly pipeline changes stay fast, approved forecasting rules stay slow, and every forecast can record what was retrieved and why. The same pattern applies to any production agent or retrieval system.

If you already run agents and cannot show where they learn, start with the free 10-question learning-loop scorecard or a Learning-Loop Audit. If the bigger issue is still the work itself, the $2,500 Diagnostic Sprint remains the right entry. Leadership teams that need a stance before the next board meeting can request the 60-minute briefing, Governing AI That Keeps Learning.

Sources: Google Research, Introducing Nested Learning. Behrouz et al., Nested Learning: The Illusion of Deep Learning Architectures, arXiv 2512.24695.

Keep reading

Related articles

Three streams of task records pass through a human approval gate before becoming lasting memory.
AI Strategy

Fast, Medium, Slow: A Memory Model for Enterprise Agent Teams.

Long-term memory for agents does not have to mean an unreviewed pile of notes. Separate this task, the daily record, and approved policy, and promote a lesson only after a person reviews it.

A presenter explains three memory layers to a leadership team in a meeting room.
AI Governance

What Google's Nested Learning Paper Means for Mid-Market Teams.

Nested Learning is a research paper, not a product. The practical takeaway for operators is that governance must keep up if AI starts learning after launch, and you can install the controls now.

An operator maps the connected stores used by several AI agents.
AI Governance

Log the Memory, Not Just the Model.

If you only version the model, you cannot explain last month's decision. Log which memory was in force, what was retrieved, and who was allowed to change it.

An operator arranges three layers of agent memory with a visible workflow between them.
AI Strategy

Lesson Promotion: The Review Step Most Agent Teams Skip.

A useful correction is not the same thing as a new policy. Name the person who decides when a lesson may move from a session into durable memory.

Related reading