Engineering AI Governance: From Policy Documents to Runtime Controls

Executive Summary

The era of treating AI governance as a policy document is over. With the EU AI Act now setting a global regulatory precedent and enforcement dates crystallizing, "we have an AI policy" is no longer a defensible governance posture — a PDF cannot redact PII, cannot enforce access control on a vector database, and cannot produce an audit trail. Governance in 2026 has to be engineered, not written.

This paper covers three controls that belong in the application architecture itself, not in a policy binder: data loss prevention at the model gateway, row-level security inside the vector database, and immutable audit logging for every generation.

Where the regulatory ground actually stands: under the EU AI Act's 2026 Digital Omnibus revision, obligations for high-risk AI systems were deferred — standalone (Annex III) high-risk systems now face binding obligations from 2 December 2027, and product-embedded (Annex I) systems from 2 August 2028 — with violations of those obligations carrying penalties up to €15 million or 3% of global annual turnover, whichever is higher, while the separate, harsher prohibited-practices tier still carries penalties up to €35 million or 7% [1][2]. A later enforcement date is not a reason to wait — it is a longer runway to build the controls properly instead of retrofitting them under deadline pressure.

1. Zero-Trust Architecture and Data Redaction

LLMs are structurally leaky. If an employee pastes customer PII, health information, or financial records into an unprotected chat interface, that data can be absorbed into a vendor's logs, caching layer, or — depending on the provider's terms — training pipeline.

A modern AI gateway enforces Data Loss Prevention (DLP) at the edge, before a prompt ever reaches an external model provider. This typically means routing every outbound prompt through an isolation layer that runs Named Entity Recognition (NER) and pattern-based detection to identify and mask or redact sensitive entities in transit.

Microsoft Presidio, an open-source framework, is a widely deployed reference implementation of this pattern: an Analyzer identifies PII entities using a combination of regex matching, NER, and context-aware detection, and an Anonymizer applies configurable transformations — redaction, replacement, masking, hashing, or encryption — before the text moves further downstream [3][4]. It supports deployment as an embedded Python library or as a standalone HTTP service, which makes it practical to insert as a gateway layer in front of any LLM provider regardless of the application's own language or framework.

2. Row-Level Security for Vector Databases

A knowledge-management AI assistant is only as secure as the permissions behind it. If a junior analyst asks a chatbot "what is the CEO's compensation," the retrieval layer must not surface HR compensation documents unless that analyst is explicitly authorized to see them — regardless of how semantically relevant those documents are to the query.

This is not automatic. Vector databases including Pinecone, Weaviate, Milvus, and pgvector all offer isolation primitives — namespaces, tenant-level shards, or row-level security policies — but none of them enforce tenant or permission isolation unless it is explicitly configured at query time [5][6]. Without it, similarity search has no concept of "who is asking": a query embedding from one user can retrieve nearest neighbors belonging to data that user was never authorized to see, purely because the two are semantically close.

Production-grade implementations require:

• Metadata-tagged access control lists (ACLs) attached to every embedded chunk at ingestion time, reflecting the same permission model as the source system.
• Identity-scoped query filtering — forcefully injecting the requesting user's identity token or role into the vector search filter at query time, so the retrieval mechanism can only "see" the subset of the index that user is authorized to access, before similarity scoring ever runs.
• Isolation architecture matched to scale — namespace-per-tenant (Pinecone's documented pattern), tenant-level shards (Weaviate), or row-level security policies (pgvector), chosen based on tenant count and query volume rather than defaulted to whatever the vector store ships with out of the box [5].

The failure mode to design against: a retrieval system that is "secure" in the sense that its API requires authentication, but whose vector search itself is permission-blind, is not secure — it is authenticated and then unfiltered. Access control has to be enforced inside the retrieval query, not just at the login screen in front of it.

3. Immutable Audit Trails

When an AI system contributes to a consequential decision — approving an insurance claim, drafting a contract clause, surfacing a candidate for an interview — the organization must be able to reconstruct exactly why, months later, under scrutiny.

Every generation that touches a business decision should be logged immutably, capturing at minimum:

• The exact prompt submitted, including any system-level instructions.
• The specific context chunks retrieved, with a version identifier for the source document.
• The model version and generation parameters (temperature, top-p) used.
• The exact output returned to the user or downstream system.

This is the same forensic-traceability standard already expected of deterministic decision systems in regulated industries — it simply has not, until recently, been consistently applied to probabilistic ones. Without it, an enterprise has no way to answer a regulator's, auditor's, or customer's most basic question about an automated decision: why did the system say that, and would it say the same thing again today?

Mapping Controls to Frameworks

Conclusion

AI governance is not about slowing innovation down — it is about building the guardrails that let it scale safely, and guardrails have to be built out of code and configuration, not prose. Data loss prevention at the gateway keeps sensitive data from ever reaching an external model. Row-level security inside the vector store ensures retrieval respects the same permissions as the source data. Immutable audit trails make every consequential AI decision reconstructable after the fact. Enterprises that engineer these three controls now — while the EU AI Act's deferred timeline still gives them runway — will meet 2027–2028 obligations as a byproduct of already having them, rather than as a scramble against a deadline.

References

1. Legalnodes, "EU AI Act 2026 Updates: Compliance Requirements and Business Risks," regulatory tracking summary, 2026.
2. DLA Piper GENIE / Inside Privacy, "The Digital AI Omnibus: Proposed Deferral of High-Risk AI Obligations Under the AI Act" and "EU AI Act Update: Timeline Relief, Targeted Simplification, and New Prohibitions," 2026.
3. Microsoft, "Presidio: Data Protection and De-identification SDK," open-source project documentation, github.com/microsoft/presidio.
4. OctaByte, "Presidio: Open-Source Framework for PII Detection, Redaction & Anonymization," 2026.
5. Pinecone, "Multi-Tenancy in Vector Databases," Vector Databases in Production series.
6. Rafter, "Vector DBs & Embeddings: The Overlooked Security Risk," 2026.
7. NIST, "AI Risk Management Framework (AI RMF 1.0)," National Institute of Standards and Technology, January 2023.
8. NIST AI Resource Center, "AI RMF Core," airc.nist.gov.
9. OWASP, "OWASP Top 10 for LLM Applications 2025," owasp.org.
10. Mend.io / Aembit, "OWASP Top 10 for LLM Applications: Risks & Mitigations," 2025–2026 guides.

Sources referenced in this paper include the EU AI Act and its 2026 Digital Omnibus amendments, NIST's AI Risk Management Framework, the OWASP Top 10 for LLM Applications, and open-source security tooling documentation (Microsoft Presidio). Regulatory dates and penalty tiers reflect public reporting as of mid-2026 and are subject to further revision; readers should consult official EU and NIST publications for authoritative, current text.

About Vibodh AI

Vibodh AI helps enterprises turn AI governance from a policy document into a running system — DLP-enforced model gateways, permission-aware retrieval architectures, and immutable audit logging built into the application layer from day one.

From AI governance readiness assessments to full runtime-control implementation aligned to the EU AI Act, NIST AI RMF, and ISO/IEC 42001, we partner with clients across industries as a long-term, responsible AI partner. Think AI. Build beyond.