AI Governance
Log the Memory, Not Just the Model.
By John Utley
Published

When a company investigates a bad AI outcome, the first question is usually which model did this. That is necessary and not sufficient. If prompts, retrieval stores, or long-term notes have changed since launch, the model identifier alone will not reconstruct the decision.
Log three things together: the model version, the memory version, and the evidence retrieved for that call. For a forecast, that means the rules and definitions in force, plus the records the system used. For a support or research agent, it means the prompt, the documents it could see, and any memory it wrote back.
This is the monitoring work already implied by NIST AI RMF Measure and Manage, and by ISO 42001 monitoring and continual improvement. It does not require a new framework. It requires a record you can actually produce.
Practical rules: version memory separately from the model. Restrict who can write to slow memory. Review those writes before they affect production. Test a rollback. Write down what would trigger a re-validation, such as a material change to retrieval sources, a spike in exceptions, or a new use case.
If you cannot show, for a past decision, which version of the model and the memory produced it, that is the gap to close first. The learning-loop scorecard asks this directly. A Learning-Loop Audit maps the stores, the write rights, and the missing logs.

