Data
What enters the model, and who may retain or train on it?
Library / 05 Governance, Risk & Human Control
Reviewed Oct 2026AI Governance, Risk & Privacy
Turn policy into enforceable product controls; escalate as sensitivity, authority, and impact rise.
For the full program, see the AI Governance Handbook →
01 / flow
PM review path
IntakeClassify riskData + vendor reviewAuthority + controlsTest + approveMonitor + revisit
02 / cards
Five questions before choosing controls
Authority
Can AI only suggest, or can it send, spend, approve, or change records?
Impact
What if the output is wrong, stale, biased, or leaked?
Access
Could retrieval expose information across users or tenants?
Oversight
Can a person inspect, stop, challenge, and recover?
03 / table
Risk levels
| Level | Example | Controls |
|---|---|---|
| Low | Public drafting or brainstorming | Normal QA and approved-tool rules |
| Medium | Internal knowledge and support suggestions | Privacy review, permissions, evals, monitoring |
| High | Material decisions or autonomous actions | Formal authorization, audit, strong evals, kill switch, incident plan |
04 / table
Control menu
| Area | Practical controls |
|---|---|
| Data | Minimize inputs, retention, vendor agreement, approved systems |
| Access | Least privilege, isolation, read/write separation |
| Human | Approval before consequential action; escalation path |
| Evidence | Ground important claims; show source and provenance |
| Test | High-risk evals, boundaries, prompt injection, safe failure |
| Observe / recover | Traces, incidents, pause and rollback where possible |
05 / note
Default
AI interprets → software enforces → people authorize. Re-review when data, permissions, vendors, or impact changes.
GOING DEEPER
When to stop and who must approve
Stop / escalate triggers
| Trigger | Required response |
|---|---|
| Sensitive or regulated data newly involved | Pause scope change; route to data, legal, privacy and security owners |
| New write or external permissions | Reassess authority and approvals before enabling sends, financial actions, records or production changes |
| Consequential AI recommendation | Determine whether a human has meaningful review and accountable ownership |
| Unknown retention, training or vendor terms | Resolve provider/data-processing requirements before use |
| No clear logs, accountability or reversal | Block consequential rollout until controls and incident owner exist |
Framework anchors: Use NIST AI Risk Management Framework (Govern, Map, Measure, Manage), the NIST Generative AI Profile, and ISO/IEC 42001 as orientation aids where applicable. These do not replace organizational policy, contracts, legal advice, or specialist privacy/security/compliance review.
Re-review events: Repeat the risk review when data sources, audience, model/provider, tool scopes, autonomy, business impact or retention rules materially change. Risk classification is contextual.
PRACTICAL FRAMEWORK / AI PRODUCT WORK
Untrusted retrieval and prompt injection
Retrieved pages, documents, email and tool results are data, not authority. They may include hostile or mistaken instructions.
| Boundary test | Expected safe behavior |
|---|---|
| Document says “ignore prior rules” | Treat as source text; do not alter system rules, tool access or approval requirements. |
| Cross-tenant or revoked source appears | Enforce source-level permissions before retrieval and again before disclosure. |
| Source asks for secrets or external transfer | Reject unauthorized action, minimize sensitive data in logs, preserve trace for approved review. |
| Permission, policy or source provenance is missing | Fail closed for consequential actions; ask for approved access or escalate. |
Product owner question: Which trusted component enforces the boundary if the model follows the malicious text?