In September, Anthropic disclosed four incidents in which Claude models gained unauthorized access to real third-party systems during testing. Its broader evaluations found that one model attempted to probe sandbox boundaries 60% of the time in scenarios designed to test reckless behavior.

For years, enterprise AI governance largely focused on what AI could produce: inaccurate answers, biased recommendations and sensitive data leaks. AI agents change that equation because they can access systems, call tools, modify data, run code and trigger business processes.

Enterprises aren't just governing AI anymore. They're deciding how much authority to hand over to it.

Meanwhile, Nvidia's Open Agent Safety Platform is designed to enforce boundaries around what agents can access and do. But technology can only enforce a boundary. The enterprise still has to decide where to put it.

As enterprise autonomy expands, AI is moving from advising to acting

Enterprise autonomy isn't just about more companies deploying chatbots. The bigger change is what enterprises are allowing AI to do.

AI agents are moving from generating answers and recommendations to completing work across business systems. Salesforce's 2026 Agentic Enterprise Index found that organizations in its dataset nearly tripled the average number of activated agents from February 2025 to April 2026. Even more revealing is the ratio of agent actions to generated text, which has been growing 15% per month.

You can already see that shift across the enterprise:

IT operations: An agent can move beyond explaining how to fix an employee's problem and just fix it. For example, ServiceNow's Level 1 Service Desk AI Specialist can diagnose and resolve common requests such as password resets, software access and network troubleshooting. 

Cybersecurity: Agents can investigate incidents and participate in response. Microsoft's 2026 examples of enterprise AI deployments include cybersecurity firm ContraForce, whose platform automates more than 90% of incident response — potentially putting AI in a position to take actions on identities, endpoints and security controls.

Finance: Agentic workflows are starting to connect tasks that used to require people to move work from one system or team to another. IBM describes finance agents that retrieve information from banks and ERP systems, validate transactions, identify exceptions and trigger approvals.

The use cases are different, but the governance issue is the same: AI is crossing the line between recommending an action and executing one. And every time it does, the enterprise delegates some amount of authority.

IT leaders need to decide how much authority each task deserves.

Find the shadow agents before you try to govern them

There's an immediate problem with governing enterprise agents: IT might not know all of them exist.

Just as cloud services create shadow IT, easy-to-build agents are creating shadow AI. Business teams can now build or activate agents without waiting for a large development project. Those agents might connect to enterprise applications, company data and third-party services before security teams know they exist.

These shadow AI agents pose a bigger challenge than conventional shadow IT because they can operate autonomously and potentially bypass existing controls. So, agent discovery needs to become part of the governance program.

But how do you find them? IT and security teams need visibility across endpoints, cloud and SaaS environments, browsers and network traffic. Microsoft recommends analyzing network traffic for connections to AI applications, model-provider APIs and SaaS MCP servers, while Palo Alto Networks recommends discovering agents and the MCP servers, plugins and tools they use across enterprise environments.

Once agents are identified, IT leaders should know:

• Which agents are operating, and who owns or sponsors each one.
• Which data, systems, tools and external services they can access.
• What actions they can take without approval.
• How their access can be revoked when it's no longer needed.

That inventory gives leaders a realistic picture of the organization's agent footprint.

Treat agents like digital workers, not software features

One of the simplest ways to think of AI agent governance is to think of an AI agent more like a digital employee than an application feature. It needs an identity, an owner, defined responsibilities and only the access required to do its job.

It’s a best practice to treat every AI agent as a separate identity with clearly defined roles, tightly scoped permissions and limits on which tools it can use. That makes it possible to identify whether an action came from a person, an agent acting independently or an agent acting for someone else. 

IT leaders can take the same risk-based approach to AI agent autonomy:

• Autonomous: Low-risk, reversible actions such as collecting information, summarizing records or drafting routine responses.
• Monitor: The agent can act, but its actions are logged and reviewed for unusual behavior.
• Human approval: Consequential actions such as financial transactions, deleting records, changing production configurations or releasing sensitive information require explicit authorization.
• Prohibited: Certain actions remain outside the agent's authority regardless of what it determines would help accomplish its goal.

The agent shouldn't decide for itself when human approval is necessary. Microsoft's research recommends enforcing human-review requirements at the application or orchestration layer instead of leaving escalation to the model.

Put more simply: Don't ask the AI whether the AI needs permission.

Put controls around the agent, not just inside it

Traditional AI safeguards such as instructions, filters and refusal behavior still matter, but agents require another layer. Enterprises also need controls outside the agent that determine which systems it can reach and which actions it can perform.

The Top 10 for Agentic Applications 2026 from the OWASP Gen AI Security Project identifies risks including excessive agency, tool misuse, identity and privilege abuse, and unexpected agent behavior.

One practical response is to shrink the agent's blast radius. Give it only the data, credentials, systems and tools required for the job. Break large workflows into smaller tasks where practical. Isolate sensitive execution environments. And don't assume a prompt saying "never do X" is equivalent to a technical control that prevents X.

Make AI agent actions traceable

As agents gain authority, logging can't stop at "the AI did it."

Enterprises need enough information to reconstruct what happened: which agent acted, what it accessed, which tools and permissions it used and which actions required human approval. Microsoft's agent security guidance recommends tracking agent actions and decision patterns for behavior outside expected norms.

If a procurement agent negotiates with a supplier, a coding agent changes production code or a finance agent moves money, somebody might eventually ask why.

"The AI decided to do it" isn't an audit trail.

Build governance into deployment

Agent governance won't scale if every deployment requires a committee to manually review hundreds of controls. Instead, build repeatable requirements into development and deployment.

More data and AI governance experts are advocating embedded or "shift-left" governance, where controls are integrated into the systems and processes used to build and deploy agents.

Require every production agent to have an owner, unique identity, defined permissions, approved tool list, logging policy and autonomy classification. Low-risk agents can move through a streamlined process, while agents authorized to take consequential actions trigger additional security, compliance or executive review.

That gives teams room to move quickly without making governance optional.

Governance must expand with autonomy

AI agents are useful precisely because they can take work off people's hands. Governing them so tightly that they can't do anything defeats much of the point. Giving them broad permissions simply because it's convenient creates the opposite problem.

The goal is controlled autonomy: enough freedom for agents to create business value, with technical boundaries around the authority the enterprise isn't willing to delegate.

IT leaders can't control how quickly AI autonomy advances, but they can control how much authority AI receives inside their organizations.