Sam Altman’s recent comments about accepting some negative consequences in exchange for artificial intelligence’s benefits have provoked an important debate about who bears the risks of technological progress. In an October 8 Guardian opinion piece, Moustafa Bayoumi challenges the premise by examining situations in which AI has allegedly contributed to serious harm, including deaths and dangerous military decisions. His central concern is that the people exposed to these risks often have little influence over the decisions that created them.
The argument raises questions that extend beyond OpenAI or any individual technology company. Artificial intelligence is moving into healthcare, education, financial services, government, customer service and increasingly autonomous operations. The organizations deploying these systems are making decisions about how much authority to delegate to probabilistic software, often before the consequences of that delegation are fully understood.
Every consequential technology involves risk, and there is a legitimate discussion to be had about how societies evaluate benefits against potential harms. The more immediate engineering question is whether organizations are doing enough to prevent foreseeable failures before those failures reach people.
For AI, that requires examining both what the system is permitted to do and how it behaves when interacting with a human being.
The Limits of Governing AI Through Its Own Intelligence
Much of the AI safety industry concentrates on the capabilities and actions of the underlying model. Developers conduct evaluations, improve training, restrict access to tools, monitor outputs and attempt to prevent systems from executing unauthorized operations. These safeguards are necessary, particularly as autonomous agents become capable of initiating transactions, accessing sensitive information and coordinating with other systems.
However, a technically compliant AI can still produce a harmful human interaction. A healthcare assistant can operate entirely within its approved information sources while responding inappropriately to someone expressing distress. A financial services agent can obey its transaction limits while misleading a customer about their options. A conversational system can remain inside its authorized role while reinforcing confusion or escalating an already difficult exchange.
These failures are difficult to address through conventional infrastructure controls because they arise within the interaction itself. Understanding what is happening requires information about the human participant, the progression of the conversation and the behavioral requirements that should apply as conditions change.
The prevailing approach often leaves those judgments to the language model. The LLM interprets the person’s emotional expression, decides which response would be appropriate and generates that response. Sometimes another probabilistic model is introduced to evaluate the result, but the underlying governance decision remains dependent on machine-generated interpretation.
That architecture creates an important limitation: The system being governed also participates in deciding whether its behavior is appropriate.
Why Independent Emotional Intelligence Matters
VERN was developed around a different architectural premise. Our Emotion Recognition System uses a deterministic neurolinguistic model that operates independently of the underlying LLM. It identifies emotional signals expressed in language and makes those signals available to a separate behavioral governance layer.
This distinction is important because emotional expression provides information that conventional agent controls generally do not capture. An increase in anger, the emergence of fear or a sustained pattern of distress can indicate that an interaction requires different handling. These signals do not establish a person’s internal psychological condition, and they should never be treated as a substitute for clinical judgment. They can, however, provide a consistent basis for applying organizationally defined behavioral rules.
With VERN OS, emotional signals can become inputs into deterministic controls governing how the AI responds, when escalation is required and which behaviors remain permissible. The enterprise establishes those requirements independently of the model generating the conversation. The underlying intelligence can reason about the task and formulate responses, while the governance layer applies the relevant rules.
This separation has implications for consistency and accountability. Organizations can examine which signals were identified, which controls were applied and how the system responded. They can evaluate whether their policies are appropriate and revise those policies without depending entirely on changes to the foundation model.
Determinism does not make emotion recognition infallible. The accuracy of the underlying model, the quality of the signals and the design of the associated behavioral controls still require independent evaluation. Its architectural value lies in making the recognition and governance process repeatable and separately testable rather than leaving those responsibilities entirely inside a probabilistic conversation.
One Governance Architecture Across Agents and Humans
The need for independent governance becomes more pronounced as AI systems grow more autonomous and interconnected. An enterprise workflow may involve several agents exchanging information, delegating tasks, accessing business systems and making recommendations before one of those agents communicates with a customer or employee.
Each transition introduces different risks. Agent-to-agent communication requires controls over authority, delegation, role containment and execution. Agent-to-human communication requires those same operational controls, along with an understanding of the human context and the organization’s standards for appropriate behavior.
VERN OS is designed to govern both through a common architecture. Deterministic Behavioral Control Modules can enforce operational and conversational requirements across agent interactions, while VERN’s independent emotional model supplies additional signals when humans become part of the workflow.
This allows organizations to establish governance policies that persist as a workflow moves between machines and people. The controls do not have to be reinvented for each model, and the standards governing human interactions do not have to depend entirely on the conversational abilities of whichever LLM happens to be deployed.
That is particularly relevant to enterprises adopting multiple models. An organization may use one foundation model for customer conversations, another for research and several specialized agents for internal operations. A model-independent governance layer provides continuity across those systems while allowing each model to be selected for its capabilities and economics.
The objective is to keep human-defined authority consistent as the underlying intelligence changes.
Accountability Must Extend to the Human Experience
The Guardian’s argument also exposes a limitation in how AI performance is commonly measured. Organizations often evaluate systems according to task completion, response accuracy, cost reduction and operational efficiency. Those metrics provide useful information, but they cannot fully describe the consequences of an AI interaction.
A customer-service agent might resolve a billing issue while leaving the customer more distressed than when the conversation began. A healthcare navigator might provide technically accurate information but fail to recognize that the interaction requires human intervention. An autonomous workflow might complete every assigned task while creating an unacceptable experience for the people affected by its decisions.
Organizations therefore need to evaluate both operational outcomes and behavioral outcomes. This requires records of what the AI attempted, what it executed, which controls were applied and how the human interaction developed over time.
VERN’s deterministic emotional intelligence and behavioral governance are intended to make that broader evaluation possible. They provide a way to incorporate measurable emotional signals and enforceable behavioral requirements into the same architecture used to control agentic operations.
These mechanisms cannot eliminate every form of AI risk. Military targeting, infrastructure security, model reliability and the societal consequences of deployment each require additional forms of oversight, testing and accountability. Human-facing behavioral governance addresses a specific and increasingly consequential part of that larger problem.
Human Control Must Be More Than a Promise
The debate surrounding Altman’s comments ultimately concerns who has authority to decide which risks are acceptable and who must live with the consequences. That is a legitimate question for policymakers, technology developers, deploying organizations and the public.
At the enterprise level, organizations already have an important responsibility within their control. They can establish enforceable standards for the AI systems they deploy, maintain independent oversight of consequential actions and determine how those systems are expected to behave toward the people they serve.
The next generation of AI governance must account for both sides of that responsibility. It needs to control what autonomous intelligence can do while incorporating an independent understanding of the human interaction into decisions about how that intelligence should behave.
This is the architecture we have spent years developing at VERN. By combining deterministic behavioral governance with an independent deterministic emotional model, VERN OS provides a common control layer across agent-to-agent and agent-to-human interactions.
The benefits of artificial intelligence may be extraordinary, but organizations should not have to surrender control over the human experience to obtain them. As AI becomes more capable and autonomous, that control becomes an increasingly important part of responsible deployment.
Human control over artificial intelligence must include control over how artificial intelligence treats humans.
Source: https://www.theguardian.com/commentisfree/2026/oct/08/ai-death-war-sam-altman
