The Governance Layer GenAI Cannot Outrun: Ensuring Evidence And Accountability

This article discusses the critical importance of governance in GenAI systems, emphasising the need for evidence infrastructure to ensure accountability and compliance. Anshul Pathak outlines strategies for integrating governance into AI workflows to support responsible deployment and operational efficiency.

A modern AI program rarely fails because the model is incapable. It fails because the organization cannot prove what the model touched, why it was allowed to touch it, and who approved the path that made it possible. When that proof is missing, every incident becomes a narrative fight: teams argue over lineage, privacy teams chase spreadsheets, and leadership is forced to slow down shipping because the system cannot answer basic accountability questions with precision.

That quiet escalation is now shaping how Responsible AI inside large enterprises. Policy is no longer the hardest part. The hard part is building an operational substrate where metadata, lineage, privacy tagging, retention rules, access controls, and audit evidence behave like production infrastructure, not documentation.

Anshul Pathak, a seasoned Software Engineer and an editorial board member of IJAIDSML, builds that substrate. His work sits in the unglamorous zone where engineering discipline decides whether Responsible AI is enforceable or performative. “If governance cannot produce evidence on demand, it is not governance,” he says. “It is policy theater, and it collapses the moment a hard question arrives.”

This matters because regulators and standards bodies have been explicit about what “good” looks like: structured risk management for AI systems, measurable controls, and repeatable governance processes rather than ad hoc reviews. NIST’s AI Risk Management Framework formalizes this expectation and frames governance as an engineering discipline that can be implemented, measured, and improved.

From Data Ownership to Verifiable Decision Paths

Enterprises used to treat governance as a front-door problem: access requests, approvals, and periodic audits. GenAI turns it into an everywhere problem. Training, evaluation, retrieval, tool use, and downstream analytics all become governance surfaces. The result is predictable: controls fragment across teams, and the organization cannot produce a single coherent story about what happened.

Pathak’s stance is that governance has to move in-path, into the execution layer, where decisions are made. That is not an abstract philosophy. It is an architectural constraint. “The moment governance lives outside the critical path, it becomes optional,” he says. “Optional controls do not survive scale.”

One reason this approach is gaining urgency is the policy environment around AI accountability. In the United States, the recent cycle of federal actions has reinforced that AI safety, security, and accountability expectations will continue to evolve and organizations need systems that can adapt rather than rewrite core workflows every time guidance changes.In the European Union, the AI Act establishes a penalty regime that makes governance failures potentially financially meaningful, not academic.

That reality changes how technical leaders should define “done.” Done is not a deployed model. Done is a deployed model with traceable lineage, enforced policy, and audit evidence that can be queried without a war room.

A System of Record for Data, Then a System of Record for ML

A practical way to understand Pathak’s work is to separate two layers that many companies blur: data governance as a foundation and ML governance as an extension. If the data layer has weak metadata, inconsistent privacy tags, or missing ownership, the ML layer inherits fragility and multiplies it.

One of Pathak’s core accomplishments has been leading the development of an enterprise-scale analytics governance platform operating across a large, globally distributed data environment. The platform governs roughly 3 million datasets across 28 organizations. Within this program, he led the design and implementation of an ML-powered data classification and tagging framework that identifies sensitive data across heterogeneous sources with 90.7% accuracy, then uses those signals to drive enforceable, in-path controls. He also advanced audit capabilities to the point where compliance workflows were reduced from weeks to minutes.

This is the work that turns governance into operational reality: a shared metadata and tagging foundation, broad coverage across teams, and audit evidence that can be produced quickly rather than reconstructed under pressure.

Building on that foundation, since 2024, Pathak has architected and implemented a federated AI and ML governance infrastructure designed to serve as a single source of truth for compliance auditing and policy enforcement across complex machine learning environments. The system governs 500+ ML workflows per day with SLO-backed policy and lineage checks, while maintaining 99.9%+ availability. Its architecture separates centralized policy definition from distributed enforcement, and a plug-in model allows new catalogs, controls, and lineage connectors to be onboarded without re-architecting the system.

This is where governance stops being a compliance function and becomes a throughput function. If governance adds unpredictable latency or brittle approvals, teams route around it. If governance is fast, reliable, and consistent, teams stop treating it as friction and start treating it as default infrastructure. “My bar is simple,” Pathak says. “Governance should not slow down engineering when the work is safe. It should make safe work the easiest path.”

The standards world mirrors this direction. NIST’s Privacy Framework pushes organizations toward measurable privacy risk management that can be integrated into enterprise processes rather than handled as one-off reviews.That is the same architectural move Pathak is describing: make controls repeatable, measurable, and embedded.

Identity for Agents: Preventing Rogue Capability Before It Exists

The newest failure mode in GenAI programs is not only data misuse. It is unbounded action. When an agent can call tools, access data, and trigger workflows, the governance question becomes identity and capability, not only data classification.

Pathak’s third accomplishment addresses that failure mode directly: a GenAI Governance Framework and Agent Identity Service that governs agent identity, data access, and interaction patterns across internal AI platforms. The service issues unique agent IDs, enforces least-privilege capabilities using scoped controls, and captures immutable audit evidence for tool calls. It has been deployed across 20 teams in 4 organizations.

This approach reflects a broader pattern in Pathak’s systems work. In a HackerNoon article describing how he re-architected a high-throughput metadata service under sustained load, he showed how pushing responsibility into the execution path and removing centralized bottlenecks allowed the system to scale without constant human intervention. The same principle applies here: governance that sits outside the critical path becomes negotiable, while governance embedded into execution scales quietly.

The value is structural: it reduces the surface area for “informal” agent behavior and creates accountability that can be proven later. “Agents should not inherit power because somebody wired a tool into a workflow,” Pathak says. “They should earn explicit capabilities, tied to identity, with an audit trail that survives scrutiny.”

Governance as Evidence Infrastructure

Pathak’s work points to a hard conclusion many organizations still avoid: Responsible AI is not a slide deck. It is evidence infrastructure. That view is codified in his book, Responsible Intelligence: Governance by Design—Building Trustworthy AI from the Data Up, which argues that accountability must be engineered into systems from the data layer through production, not bolted on after deployment.

The outcome is not simply compliance. It is speed with accountability. “When the foundations are correct, teams move faster because they do not have to negotiate governance every time,” Pathak says. “They can focus on building, while the platform enforces the rules consistently.”

If GenAI is going to scale safely inside large enterprises, this is the layer it will ultimately depend on: not model intelligence, but the ability to prove control, explain decisions, and sustain accountability as systems grow.