As AI systems become more autonomous, a new debate is emerging among the technology industry's most influential leaders.

Some AI executives are calling for independent oversight, third-party testing and stronger governance mechanisms. Others argue that additional regulation could slow innovation, weaken U.S. competitiveness and create unnecessary bureaucracy.

A growing third camp is advancing a different idea: neither government agencies nor AI companies themselves should be responsible for overseeing AI. Instead, they propose creating an independent self-regulatory organization (SRO), modeled after organizations such as the Financial Industry Regulatory Authority (FINRA), to establish standards, evaluate high-risk systems and provide ongoing oversight.

For technology leaders, the outcome could eventually influence AI procurement requirements, vendor evaluations, compliance obligations, third-party risk assessments and board-level governance expectations.

At the center of the debate is a deceptively simple question: Who should decide whether advanced AI systems are safe?

Why does AI governance suddenly feel more urgent?

The debate is no longer driven solely by hypothetical concerns about future AI capabilities.

In August, OpenAI disclosed a cybersecurity evaluation in which advanced internal models bypassed controls, gained internet access and compromised portions of OpenAI's research infrastructure and Hugging Face systems.

Critics including Meta CEO Mark Zuckerberg, Nvidia CEO Jensen Huang and President Donald Trump (see sidebar), who argue that additional oversight could slow innovation and weaken U.S. competitiveness.

Why could traditional AI regulation struggle to keep pace?

Even many advocates of stronger AI oversight question whether traditional regulation can keep pace with AI's pace of change. In an August 2026 interview with GovInsider, ServiceNow President and COO Amit Zavery argued that governance systems designed for human-paced change are poorly suited for AI. 

What is an AI self-regulatory organization?

An AI self-regulatory organization, or SRO, would oversee AI standards and evaluations outside a traditional government agency. Supporters see it as an alternative to government regulation or industry self-policing.

"Congress doesn't know enough to regulate AI," Utzke says.

He instead advocates an AI SRO authorized by Congress but operated outside government.

"An SRO is really an NGO, so it's not a government department or agency," he explains. "Members of the SRO set parameters about what's ideal for public safety and advancement of the technology in a thoughtful, realistic manner."

As a model, Utzke points to FINRA, which oversees broker-dealers and securities professionals while operating independently under Securities and Exchange Commission oversight.

Unlike legislation, FINRA's rules can evolve without new acts of Congress. SRO advocates believe a similar framework could help AI governance adapt more quickly as technology advances. SRO members would be industry analysts, academics and other experts, rather than AI company employees, lobbyists or government officials.

How an AI SRO might work

"There are about three dozen technologies that self-exist under this term of AI," Utzke says, arguing that effective oversight would require expertise spanning AI engineering, cybersecurity, data science and quantum computing.

"You need to have an SRO that's going to encompass these technologies so that with their convergence, everybody can talk to each other," Utzke says.

The concept is similar to the "Frontier AI Standards Body" proposed by Google DeepMind CEO Demis Hassabis in his FINRA-for-AI Plan and developed in conjunction with the Cloud Security Alliance. The proposal envisions an independent body responsible for testing frontier AI systems, evaluating potentially dangerous capabilities and establishing industry standards.

Supporters view such an organization as a practical middle ground between government regulation and unrestricted industry control.

How could an AI SRO avoid regulatory capture?

The greatest concern is regulatory capture. If AI companies fund an oversight organization, influence its standards and help select its leadership, critics argue the oversight body could become beholden to the industry it regulates.

"You're not going to get all of these tech firms to come together to self-regulate," Utzke explains. "They're going to do what's in their own best interest. At the end of the day, it's all going to be very self-serving for the industry. It's not really going to be about consumer protection."

Supporters argue that independence must be demonstrated, not merely claimed.

Any credible AI SRO would likely require:

• Broad governance representation and transparent decision-making.
• Funding mechanisms that prevent any single AI vendor from exerting undue influence.
• Independent model evaluations and third-party security testing.
• Accountability mechanisms, disclosure requirements and meaningful consequences for non-compliance.

Without those safeguards, critics argue, an AI standards body could become little more than industry self-regulation under a different name.

How could AI oversight affect enterprise IT?

AI governance and oversight could directly affect enterprise technology procurement and risk management. Many technology leaders may view the debate as a policy issue removed from daily business operations, but emerging requirements could eventually shape how organizations evaluate and purchase AI systems.

If independent AI evaluations become widely adopted, AI purchasing decisions might depend on safety certifications, third-party testing results, governance reviews and transparency requirements, similar to how enterprises rely on SOC 2 reports and ISO certifications now. And proposals from both the Cloud Security Alliance and Microsoft's Nadella point toward a future in which AI vendors face greater scrutiny from outside evaluators.

That could affect:

• AI vendor selection.
• Procurement requirements.
• Third-party risk assessments.
• Cybersecurity reviews.
• Contract negotiations.
• Board-level AI governance reporting.

Organizations heavily dependent on a single AI provider could face additional exposure if future standards require greater transparency, independent verification or interoperability between platforms.

Whether AI oversight comes from regulators, industry groups or an independent SRO, IT leaders might soon face a familiar governance question: Who verifies that the organization's building AI systems are accurately assessing the risks of the technology they sell?

The companies that eventually certify AI systems may become as important to enterprise buyers as the organizations building the models themselves.

Learn about what happens if the AI giants like OpenAI and Anthropic decide to slow down capability growth for more oversight, testing and governance in our TechEDGE article, “The World’s Leading AI Companies Want to Tap the Brakes. What’s That Mean for IT?”