Enterprise AI programmes often draw a roadmap: build fast, deploy to a friendly user group, then add governance — logging, model risk paperwork, red-teaming, residency hardening — in phase two.
Phase two rarely arrives before audit, a regulator question or an incident does. By then retrofitting controls costs more than building them in, and trust is already damaged.
Governance is not a phase. It is part of the product specification from discovery onward.
What governance means in practice
Not a policy PDF in a drawer. Concrete controls:
- Intended use and limits — what the system may and may not do
- Human oversight — approval gates, sampling review, escalation queues
- Logging — attributable record of inputs, tool calls, outputs, model version
- Data handling — residency, retention, no training on client data
- Evaluation — including compliance and adversarial scenarios
- Model risk documentation — limitations, monitoring plan, change control
- Incident response — runbooks, rollback, comms
Each item maps to implementation tasks in the build backlog, not a future programme.
Why phase two fails
Momentum narrative. Teams celebrate launch and move to the next feature. Governance work competes with roadmap and loses.
Ownership gap. Compliance consulted late lacks bandwidth to retrofit. Engineering resists changes that touch core flows.
Technical debt. Adding logging comprehensively after agents go live requires refactoring orchestration. Skipping is tempting.
Vendor departure. Pilots built by vendors who leave rarely hand over evaluation suites or control documentation.
Discovery sets the baseline
The discovery pack includes a governance and security baseline tailored to the use case: data flows, provider choices, oversight model, logging scope, red-team plan, evaluation categories for policy.
Architecture review and security sign-off start from that document. Open questions are listed explicitly — not discovered in production.
Human oversight by design
"Human in the loop" is not a banner. It is a state machine:
- Auto-execute below confidence threshold X for action class A
- Draft and wait for approval for action class B
- Never automate action class C
Documented, implemented, tested in evaluation.
Regulators and internal audit ask who approved automated decisions. You need names, rules and records — not "users can always override."
Logging that auditors can use
Defaults vary by client. Common pattern:
- Log action, actor, timestamp, outcome, model version
- Log tool names and parameters for writes; avoid full prompt content in logs unless audit requires it
- Retention aligned to policy
- Access to logs restricted and itself audited
Agent action logged and attributable is a company standard, not a nice-to-have.
Red-team before scale
Red-teaming finds prompt injection, data exfiltration paths and policy bypasses before adversaries or journalists do.
Findings become regression tests. Red-team is not a one-off theatre exercise before launch; it repeats when models, tools or corpora change materially.
Third-party models
When hosted models are in scope, discovery documents provider data handling, network paths, subprocessors and fallback options. On-premise or private deployment remains available where policy requires.
We do not invent certifications. We implement controls and provide documentation your frameworks need.
Governance in managed run
Systems evolve. Models upgrade. Corpora refresh. New regulations appear.
Managed run includes quarterly roadmap review — evaluation updates, control adjustments, evidence for ongoing audit. Governance is continuous, not a gate passed once.
Questions for your steering committee
- Where is the governance baseline document for the current AI work?
- Which releases were blocked by evaluation failure — ever?
- Show one complete audit trail for an automated action last week.
- Who owns model risk after go-live?
- What triggers a governance review when the model or corpus changes?
Silence on any row is a programme risk.
Internal audit cadence
Schedule audit sampling before annual audit season. Provide eval results, change logs, sample action trails and oversight metrics quarterly.
Proactive evidence reduces fire drills and builds trust with risk committees weighing second and third use cases.
Regulators and external scrutiny
When external scrutiny applies, discovery documents intended use boundaries and monitoring. Material changes trigger reassessment — new geography, new data type, new autonomous action class.
We do not speak for your regulator. We build evidence your legal and compliance teams can use in their filings and responses.
Aligning with existing IT controls
AI systems should plug into existing change management, access review and incident processes — not parallel shadow IT. Discovery maps which enterprise controls apply and where AI-specific controls add.
Model risk as living documentation
Model risk docs are not one PDF at launch. They update when intended use expands, models change, new data types enter retrieval or autonomy increases. Managed run includes doc refresh in quarterly roadmap.
Version model risk docs alongside model and prompt versions in git or your GRC tool.
Segregation of duties for agents
Agents that read and write need SoD mapping: who configures tools, who approves production prompt changes, who reviews samples, who holds break-glass credentials. Discovery workshops surface SoD gaps before automation enforces them wrongly.
Third-party and subprocessors
When hosted models process data, subprocessors list must be accurate for your vendor risk programme. We document providers used in architecture and notify when they change materially.
How Incubics builds
Standards from site and method:
- Data residency by region
- Every agent action logged and attributable
- No client data used to train shared models
Governance from day one is a why point, not marketing. It is sprint one work.
Risk and compliance teams have a dedicated For your role page. FAQ covers IP, residency and security review.
If your build plan has governance in phase two, rename phase two to sprint one or accept the risk you are carrying.
Practical governance artefacts
Keep artefacts concise: one-page intended use summary for execs, detailed control matrix for risk, runbook for ops.
Map controls to tests — each control should link to eval case or monitoring alert where possible.
Privacy impact assessment timing varies by region — discovery notes trigger points for your legal team.
Board AI policies often lag technology — governance pack helps policy catch up with evidence.
Whistleblower and user reporting channels for harmful AI outcomes — comms should mention them where applicable.
Vendor model changes notified to risk team — subprocessors and behaviour shifts may require reassessment.
Joint steering with compliance quarterly after launch — standing agenda item, not ad hoc panic.
Governance debt tracked like tech debt — visible backlog with owners.
Good governance enables speed — clear guardrails let teams ship inside boundaries confidently.
Phase two governance is a myth that expensive incidents disprove.
Closing note
Production AI is a programme of small disciplined choices: discovery before build, evaluation before users, APIs before agents, runbooks before scale, adoption alongside code. Skip any one and the demo survives while the business outcome does not.
Incubics works that way by design — two-week discovery, production by week twelve, optional managed run. Offices in Bengaluru, Pune and the USA. Legal entity IQLEXA Technologies Private Limited.
If this piece matches a problem you are living, the next step is a conversation, not another pilot.
hello@incubics.com for discovery that produces a baseline your compliance team can actually use.