Risk data · Lineage evidence
MixedRisk and financeData lineage a supervisor can follow: questions risk and finance teams ask
Supervisors keep asking banks to show where a risk number came from. These field notes answer the questions risk and finance teams raise when they try to keep that evidence current, and where Agent Serve fits.
Valnora & Elric team6 min read
Risk and finance teams hear the same request from supervisors in different words: show us where this number came from. The Basel Committee’s principles for risk data aggregation and risk reporting, known as BCBS 239, set the expectation more than a decade ago. In 2024 the ECB published a guide that spells out what it expects on risk data, including complete and up-to-date lineage for the risk indicators in scope.
These notes collect the questions teams ask when they try to keep that lineage evidence current, and answer them plainly.
What do supervisors actually mean by lineage?
Lineage is the path a number takes from where it was captured to where it is reported: the source system, every transformation and join along the way, and the controls applied at each step. The ECB guide describes it as information about how data moves and changes from capture to report.
In practice, a supervisor wants to pick a figure from a regulatory or board report and see, without a project to answer it: which systems fed it, how it was calculated, who owns it, which checks ran, and when someone last confirmed all of that is still true.
The hard part is keeping it true, because the warehouse changes faster than the documentation. A view gets a new join, a source system adds a field, a calculation is fixed after a reconciliation break. Each change is reasonable. None of them updates the lineage document, which was written for the last review and sits in a shared drive.
The fix is not a better document. It is a register that is checked against the real definitions on a schedule, with someone responsible for every gap it finds.
What should the lineage register hold?
Keep it in a table, one row per critical data element per report. A useful row has:
- Element and reports: what it is and every report that uses it.
- Source and hops: the system of capture and each view or table on the way.
- Definition: a link to the governed definition, not a paragraph of prose.
- Controls: the reconciliations and quality checks that apply, and where their results live.
- Owner and review date: who confirms it and when they last did.
What the solution produces
Lineage register row (illustrative)
Record- Data element
- Exposure at default, corporate portfolio
- Used in
- Credit risk report to the board; internal capital report
- Source
- Loan system facility table, nightly extract
- Hops
- Staging view → credit risk mart → EAD calculation view → report table
- Definition
- Governed metric “EAD corporate”, semantic model entity Facility
- Controls
- Daily reconciliation to the general ledger; completeness check on facility count
- Owner
- Head of credit risk data
- Status
- Changed hop detected in EAD calculation view · review due
How do governed definitions help?
Half of all lineage questions are really definition questions: which “exposure” is this. In Agent Serve, the semantic model describes the business entities, how they join and which measures exist, and governed metrics define each measure once. When a dashboard tile, a report extract and a Relay all read the same metric, the register can point to one definition instead of three.
Metrics in Agent Serve cover aggregates, weighted measures, ratios and formulas. Time-intelligence measures such as month to date are not built yet, so keep those calculations in the warehouse and register them as hops.
How does the register stay current?
Keeping lineage evidence current
Data · Register and definitions
- 01Lineage registerOne row per critical data element and report: source, hops, owner, controls, last reviewed.
- 02Semantic modelEntities, joins and measures defined once, so a report figure maps to a named definition.
- 03Gap rulesFlags elements with no owner, an unreviewed change or a failed reconciliation.
Workflow · Metadata refresh
- 04Nightly scheduleA Schedule block starts the refresh after the overnight data loads.
- 05Read warehouse metadataSnowflake, BigQuery or database blocks read table and view definitions in scope.
- 06Compare with the registerChanged definitions are compared with recorded hops and written as change rows.
Relay · Data Steward
- 07Daily review at 07:00The Data Steward template, in Observe, reads the register and gap flags each morning.
- 08Gap briefingA short list of new gaps and changed hops, posted to the data governance lead’s chat.
- 09Owners fix and attestData owners update the register and record their review. The Relay does not write table rows.
Dashboard · Lineage coverage
- 10Coverage by reportShare of elements with complete lineage, owner and current review, per regulatory report.
- 11Gap monitorAlerts when gap rules match more elements than the agreed tolerance.
A workflow runs nightly after the data loads. It reads table and view definitions for the objects in scope through Snowflake, BigQuery or database blocks, compares them with the hops recorded in the register, and writes any difference as a change row for the owner. It does not edit the owner’s record of the lineage itself.
Rules then flag gaps: an element with no owner, a change row older than the review window, a reconciliation that failed last night. A monitor alerts the data governance lead when the number of flagged elements passes the tolerance you agree for each report.
Each morning, a Relay started from the Data Steward template, which runs daily at 07:00 at the Observe autonomy level, reads the register and gap flags and posts a short briefing into the governance lead’s chat: what changed, what is now overdue, which reports are affected. It reads and reports. Writes to tables would go through a workflow it runs, and in Observe its writes, email included, are blocked.
What does the dashboard show a supervisor?
A lineage coverage dashboard sits on the register. For each report it shows the share of elements with complete hops, a named owner and a review inside the window, plus open gaps by age. Drill-down goes from report to element to the change rows behind a gap.
Row-level policies can limit each owner to their own elements while the CRO’s office sees everything, and a scheduled monthly PDF gives the risk committee the same view.
For evidence that the process itself ran: every workflow run is traced block by block in Logs, with inputs, outputs and errors, so you can show which definitions were read on a given night and what was compared. Relay activity and approvals go to the audit log, which admins can export. Workflows run from a deployed version, so a change to the refresh logic is reviewed first.
Controls and governance
Run logs
Every workflow run records each block’s inputs, outputs and errors, so a reviewer can trace how a result was reached.
Audit trail
Proposals, approvals, declines and executed actions are written to the workspace audit log, which admins can export.
Row-level access
Row-level policies narrow what each person sees in tables, dashboards, exports and scheduled reports.
Roles and permission groups
Admin, write and read roles per workspace, plus permission groups that limit which models, blocks and integrations a team can use.
Deployed versions
Workflows run from a deployed version, so a change is reviewed before it reaches production runs.
Self-hosting
Run Agent Serve in your own cloud or data centre, next to the systems it reads.
SSO and SCIM
Sign in through your identity provider with SAML 2.0 or OIDC, and provision people with SCIM.
More on how Agent Serve handles access, data and audit on the security page.
Where should a team start?
Pick one report that supervisors ask about most, often a credit risk or liquidity report. List its critical data elements with the owners, build the register for those only, and run the nightly refresh for a month. If the change rows match what actually changed, add the next report.
Lineage feeds directly into regulatory reporting prep, and the same register discipline applies to model risk documentation.
Questions teams ask
Is Agent Serve a lineage scanning tool?
No. It does not parse your ETL code into a full lineage graph. It keeps the register, refreshes what it can read from warehouse metadata, flags gaps and keeps the review record. If you already run a lineage scanner, read its output into the register instead.
Can a model update the register on its own?
The refresh workflow writes detected changes as change rows. Owners confirm them. The Data Steward Relay only reads and reports; any write to a table would go through a workflow it runs, and at Observe its writes are blocked.
What you need
- A list of the regulatory and board reports in scope
- The critical data elements behind each report, agreed with their owners
- Read access to warehouse metadata, such as Snowflake or BigQuery information schemas
- Existing reconciliation and data quality results the register can point to
- Named owners for each element who will review changes
- An agreed tolerance for open gaps per report