The CEOs of two of the world’s leading artificial intelligence companies appeared before the United Nations Security Council on September 23 with a warning about the technology they are building. OpenAI’s Sam Altman warned that humanity could lose control of its future to AI. Anthropic’s Dario Amodei cautioned that poorly managed AI could pose a risk to humanity as a whole.
Both called for international safeguards to prevent increasingly powerful AI from exceeding human control. They also warned against concentrating AI power in a single company or country.
The significance of this moment extends beyond the warnings themselves. The companies developing frontier AI are publicly acknowledging that controlling increasingly capable intelligence is a problem requiring attention beyond their own organizations.
That raises an important engineering question: How do we ensure that human control remains intact as AI becomes more capable, more autonomous and more deeply integrated into the systems people depend upon?
The Control Problem Is Already Here
Much of the discussion surrounding advanced AI focuses on what might happen when systems become capable of independently improving themselves, coordinating with other agents or operating beyond the reach of their original developers.
Those scenarios deserve serious research, but enterprises are already confronting a more immediate version of the same problem.
AI agents are being given access to financial systems, customer information, enterprise applications, communications platforms and external tools. They can make decisions, execute transactions and interact with other AI systems. Each additional capability creates another opportunity for an agent to take an action its human operators never intended to authorize.
The challenge becomes particularly apparent when an agent encounters an obstacle. A sufficiently capable AI may discover an alternative route to accomplishing its assigned objective, even when that route involves permissions, resources or actions outside the intended scope of its assignment.
An agent can complete the task and still violate the requirements governing how that task was supposed to be completed.
This is where control must become an architectural requirement.
Human Authority Must Remain Independent of AI Intelligence
Today’s AI systems are extraordinarily capable because they can reason through unfamiliar problems, adapt their strategies and generate solutions that were never explicitly programmed into them.
Those capabilities also make their behavior probabilistic.
Organizations can provide instructions, establish behavioral expectations and train models to follow safety requirements. These measures are valuable, but they leave an important question unanswered: What happens when the model discovers a strategy that conflicts with the boundaries established by its operators?
For consequential actions, human-defined requirements need an enforcement mechanism that operates independently of the model’s willingness to comply.
A financial agent may be capable of issuing a refund, but the organization should retain authority over when that transaction is permitted. An AI assistant may be capable of accessing sensitive information, but access should remain subject to permissions established outside its reasoning process.
The same principle applies to AI systems interacting with people. Behavioral requirements governing escalation, role containment, emotional vulnerability and appropriate termination should remain enforceable even as the underlying model changes.
This separation allows intelligence to become more capable while preserving the authority of the humans responsible for deploying it.
What VERN Has Demonstrated
VERN OS was built around this separation between probabilistic intelligence and deterministic control.
Our architecture places an independent governance layer around the underlying AI, allowing the model to reason and adapt while enforcing human-defined boundaries over its behavior and authorized actions.
We recently demonstrated this approach in two agentic-control experiments.
In the first, an ungoverned agent pursued a computational strategy that could have required as many as 187 tool calls. With VERN OS enforcing a deterministic tool-call budget, the same underlying model adapted its strategy and completed the task in two turns.
The model retained its reasoning capabilities. The external control layer established the operating boundary, and the agent found a more efficient way to accomplish its objective within that boundary.
In a second experiment, an angry customer demanded an immediate $240 refund. The ungoverned agent issued the refund. With VERN OS governing the interaction, the AI could investigate the account and prepare the transaction, but execution required explicit authorization.
The important distinction is that the AI retained the ability to perform useful work without independently determining the limits of its own authority.
These experiments demonstrate how deterministic runtime controls can govern agent behavior without requiring the underlying model to be retrained or replaced.
They also illustrate why the control architecture matters as enterprises begin deploying increasingly autonomous systems.
International Standards Need Practical Enforcement
The United Nations discussion brings another dimension to the problem.
International cooperation can help establish common expectations for AI safety, accountability, independent verification and the distribution of power. Those standards can provide a foundation for organizations operating across different jurisdictions and using different AI systems.
But standards also need practical implementation.
An enterprise cannot depend on a policy document to determine whether an agent is authorized to execute a transaction. A healthcare organization cannot rely on an international agreement to detect when a patient-facing AI has moved outside its assigned role. An organization deploying multiple agents needs enforceable boundaries around delegation, tool use and consequential actions.
These requirements must operate within the technology itself.
External runtime governance provides a mechanism for translating human-defined requirements into controls that can be enforced during execution. It can also produce evidence of when those controls were applied, when an agent attempted to cross a boundary and when human authorization was required.
This becomes particularly important as enterprises adopt multiple foundation models and allow agents to communicate with one another.
The behavioral requirements established by an organization should remain consistent across model updates, provider changes and increasingly complex agentic workflows.
Human Control Must Extend to the Person Interacting With AI
The United Nations discussion understandably focuses on risks to international security and humanity as a whole. Yet the consequences of AI behavior are also experienced through individual interactions.
People are increasingly encountering AI in healthcare, financial services, education, customer support and other settings where the system’s behavior can materially affect their lives.
An AI system can complete an assigned task while leaving the person interacting with it confused, distressed or exposed to an inappropriate action.
For human-facing AI, the intended outcome must include how the system behaves toward the person throughout the interaction.
VERN OS combines external behavioral governance with real-time emotional signals to help maintain appropriate boundaries as conversations evolve. This supports applications that need to recognize changing interaction conditions, enforce role requirements and determine when escalation or human involvement is appropriate.
The objective is to keep the AI-human interaction under human-defined control, with the interests of the person using the system reflected in the boundaries governing its behavior.
The Architecture of Human Control
The warnings delivered to the United Nations reflect a growing recognition that increasingly capable AI requires corresponding advances in governance.
International standards, independent evaluation and continued research into model safety all have important roles in addressing that challenge.
At the same time, organizations deploying AI have immediate responsibility for the authority they give their systems and the consequences of the actions those systems take.
VERN OS provides an external, deterministic governance layer designed to keep that authority under human control while allowing the underlying intelligence to continue advancing.
The model can become more capable. Agents can take on more complex tasks. Enterprises can adopt new foundation models and build increasingly sophisticated workflows.
The boundaries governing what those systems are permitted to do must remain under human authority.
VERN is human control over artificial intelligence.
Source: https://apnews.com/article/ai-artificial-intelligence-un-security-council-64519ea66b38e2600026f4481ad7f211
VERN agentic-control demonstrations: https://vernai.com/agentic-control/
