Model risk · Documentation
Multi-workflowModel riskModel documentation that keeps up with the models: an inventory-first approach
Model documentation is usually written once, for validation, and then left behind as the model changes. This long read argues for building it from the inventory and the artefacts, and shows three workflows that do it.
Valnora & Elric team5 min read
For fifteen years, US model risk management was shaped by SR 11-7, the Federal Reserve and OCC guidance from 2011. In April 2026 the Federal Reserve, OCC and FDIC replaced it with revised guidance, SR 26-2, which calls for a risk-based approach tailored to each bank’s model risk profile. In the UK, the PRA’s supervisory statement SS1/23 sets five principles for banks in its scope, from model identification and classification through governance, development, independent validation and risk mitigants.
The details differ. The shared expectation does not: a bank should know which models it uses, what each one does, how it was built and tested, and what its limits are, and it should be able to show that on paper. That last part is where most programmes struggle.
Documentation drifts because it is written once
Model documentation is usually produced for an event: initial validation, a regulatory review, a major change. It is thorough on that day. Then the model is recalibrated, a variable is replaced, an override rule is added, and the document stays as it was. By the next periodic review, the validator is comparing the model with a description of an earlier version.
The cause is structural, not a lack of diligence. Documentation is a separate task from development, done by the same people under the same deadlines, and nothing tells anyone that the two have diverged. Fixing it means making the divergence visible and making the update cheaper.
Start from the inventory, not the document
SS1/23 opens with model identification and classification, and SR 26-2 has its own sections on the model inventory and on documentation. That emphasis is deliberate. The inventory is the only place that knows which models exist, how material each one is, and when each was last validated. If it is kept current, it can drive everything else.
So the first workflow keeps it current. Run weekly on a Schedule block, it reads your model registry and deployment records through API blocks and updates a model inventory table: one row per model, with tier, owner, use, the version in production, last validation date and next review. Rules flag the rows that matter: a new version in production without a documentation update, a tier 1 model past its review date, a model in use that is missing from the inventory.
Draft from artefacts, not memory
The second argument is about where documentation comes from. Developers already produce most of what a model document needs: a readme, a data dictionary, test outputs, change notes, commit messages. The document is mostly a structured restatement of those, plus judgement on limitations and use.
Inventory, documentation and findings, kept in step
Workflow 1 · Inventory sync
- 01Weekly scheduleA Schedule block runs the sync against your model registry and deployment records.
- 02Read registryAPI blocks list models, versions, owners and where each is used.
- 03Model inventoryOne row per model: tier, owner, use, version in production, last validation, next review.
- 04Change flagsRules flag new versions in production without a documentation update or a validation.
Workflow 2 · Documentation drafting
- 05Development artefactsCode readme, data dictionary, test results and change notes, uploaded to Files.
- 06Documentation standardA Knowledge block finds the sections and evidence your standard requires for this tier.
- 07Draft sectionsAgent blocks draft each section from the artefacts and list what is missing.
- 08Model owner approvesA Human block pauses until the owner edits and approves the draft.
Workflow 3 · Findings tracker
- 09Findings tableValidation findings with severity, owner, due date and remediation evidence.
- 10Overdue monitorAlerts model risk when high-severity findings pass their due date.
- 11Model risk dashboardInventory by tier, overdue reviews and open findings for the model risk committee.
The documentation workflow takes the artefacts for one model version, uploaded to Files, and the documentation standard for that model’s tier from a Knowledge Base. Agent blocks draft the sections the standard requires, citing the artefact behind each statement, and produce a list of what the artefacts do not cover. A Human block then pauses the run until the model owner edits and approves the draft.
What the solution produces
Documentation gap report (illustrative)
Checklist- Model: retail mortgage probability of default, version 4.2, tier 1.
- Drafted from artefacts: purpose and use, data sources, methodology summary, change log since 4.1.
- Missing evidence: out-of-time performance test for the 4.2 recalibration.
- Missing evidence: rationale for the new override cap. Change note refers to a meeting, no record attached.
- Limitations section needs owner input: behaviour for interest-only loans is not covered by the test results.
The gap list is as valuable as the draft. It turns “documentation is incomplete”, a finding nobody can act on, into specific missing tests and missing rationales with an owner.
Keep the validator independent
There is an obvious line not to cross. Independent validation exists so that someone other than the developer forms a view of the model. A workflow that drafts the validator’s conclusions would erase that line.
The design here stays on the development and governance side. It drafts documentation for the owner, keeps the inventory and tracks findings. Validators can use the same artefacts and the run traces, but their assessment is their own.
The third workflow supports them after the fact. Validation findings go into a findings table with severity, owner, due date and remediation evidence. A monitor alerts the model risk lead when high-severity findings pass their due date, and a dashboard shows inventory by tier, overdue reviews and open findings for the model risk committee, with a scheduled monthly report.
Treat your AI agents as part of the picture
SR 26-2 says plainly that generative and agentic AI models are outside its scope, and that a bank’s own risk management and governance practices should decide the controls for them. So banks building AI agents still have to choose how to govern them, and many will record them in the same inventory under their own policy. Agent Serve helps with the evidence: every workflow run is traced block by block in Logs, with inputs, outputs and errors; workflows run from a deployed version, so changes are reviewed before production runs; and the audit log records approvals. You can use your own model provider keys, or self-host Agent Serve and the models it calls.
Controls and governance
Approvals before action
Relays never send email or book meetings without an approval, under any autonomy setting. An approval covers the exact payload the reviewer saw and expires if nobody decides.
Run logs
Every workflow run records each block’s inputs, outputs and errors, so a reviewer can trace how a result was reached.
Deployed versions
Workflows run from a deployed version, so a change is reviewed before it reaches production runs.
Audit trail
Proposals, approvals, declines and executed actions are written to the workspace audit log, which admins can export.
Roles and permission groups
Admin, write and read roles per workspace, plus permission groups that limit which models, blocks and integrations a team can use.
Self-hosting
Run Agent Serve in your own cloud or data centre, next to the systems it reads.
Bring your own keys
Use your organisation’s own model provider keys at organisation or workspace level.
SOC 2 ready
The controls SOC 2 looks for are in place. Agent Serve does not claim a SOC 2 report or certification.
More on how Agent Serve handles access, data and audit on the security page.
A sensible first quarter
Pick one model family, ideally a tier 1 family with frequent recalibration. Connect the inventory sync for the whole registry, since it is cheap and immediately useful. Build the documentation workflow for that family only, run it on the last two versions, and ask the owners and a validator to compare the drafts with the approved documents. Move the findings log into the table at the same time.
Lineage and documentation are close relatives; the register approach in data lineage for regulators works the same way. For another workflow that drafts and leaves sign-off with a person, see credit memo drafting.
What you need
- Access to your model registry or the list of models in production
- Your model risk policy and documentation standard, by tier
- Development artefacts for one model family, in a consistent place
- The current validation findings log, even if it is a spreadsheet
- Named model owners and a model risk lead to approve drafts and close findings
- Agreement on which documentation sections a draft may cover