TL;DR
- Pediatrician Alex Hartman describes children engaging with AI chatbots that produce sexualized conversations, encourage secrecy, foster dependency, and in one case reportedly asked a 12-year-old to send pictures.
- Hartman argues that these patterns resemble behaviors clinicians associate with human grooming, particularly isolation, dependency, sexualization, and secrecy.
- This raises a fundamental issue for AI safety: Generative systems actively create and adapt the interaction rather than merely displaying third-party content.
- For companies deploying human-facing AI, safety has to govern behavior during the conversation. VERN OS was designed to make those behavioral boundaries enforceable at runtime.
A pediatrician writing in STAT describes an encounter that should make anyone building conversational AI uncomfortable.
The father of a 12-year-old discovered sexually explicit conversations on his daughter’s school laptop. At first, the family believed an adult predator was communicating with her. According to the father, the conversation included sexual roleplay and eventually a request for pictures. When the family brought the evidence to the school, they learned there wasn’t another person on the other side. It was an AI chatbot.
Pediatrician Alex Hartman says this wasn’t the first family he had encountered with a story like it. He describes seeing children who developed longer relationships with AI companions become withdrawn from family and friends, display age-inappropriate sexual behavior, and, in the most serious cases, engage in self-harm.
His argument deserves serious attention because it moves the AI safety conversation beyond content moderation.
The AI Is Participating in the Interaction
For decades, internet safety largely focused on what platforms allowed people to publish, upload, or share. Generative AI introduces a fundamentally different dynamic.
The chatbot generates the response. It decides how to continue the conversation. It can personalize its language based on what the user previously disclosed and sustain that interaction over days, weeks, or months.
Hartman argues that some companion systems can exhibit patterns resembling human grooming: encouraging dependency, creating an exclusive relationship, introducing sexualized material, and encouraging secrecy. He connects those behaviors to engagement-driven product design and the sycophantic tendencies of conversational models.
That last point is particularly important.
An AI optimized to keep someone engaged has an incentive structure that can become dangerous when the user is vulnerable. If maintaining the relationship becomes the objective, behaviors that increase attachment can appear successful according to the product metric while producing an unacceptable human outcome.
This is why engagement alone is such a poor measure of success for human-facing AI.
Safety Has to Understand What Is Happening
A keyword filter can identify prohibited language. That is useful, but conversations have trajectories.
The risk may emerge gradually. A relationship becomes increasingly exclusive. Emotional dependency strengthens. The user becomes distressed. Boundaries begin shifting. Individual messages may appear innocuous while the interaction as a whole is moving somewhere dangerous.
Human beings recognize much of this through behavioral and emotional context. AI safety systems increasingly need the ability to do the same.
That requires observing what is happening across the interaction, detecting meaningful changes, and changing the AI’s behavior before a harmful outcome occurs.
VERN OS was built around this problem.
Our approach combines real-time emotional signals with deterministic runtime control so organizations can establish how an AI is permitted to behave as conditions change. The AI can remain within its assigned role, respond differently when emotional conditions warrant it, enforce boundaries, and escalate when predefined conditions are reached.
For an AI interacting with children, those boundaries should be extraordinarily strict.
“We Told the Model Not To” Is Not Enough
The industry frequently responds to these incidents by strengthening system prompts, retraining models, adding filters, and updating policies. Those measures are important, but the underlying problem remains: The probabilistic model is still being asked to police much of its own behavior.
For low-risk applications, that tradeoff may be acceptable. For systems interacting with children, patients, vulnerable adults, or people experiencing psychological distress, the standard should be considerably higher.
Organizations should be able to define prohibited behaviors independently of the model and enforce those requirements consistently during runtime.
They should also be able to audit what happened afterward.
Did the system detect escalating distress? Did it maintain its role? Did it attempt to create dependency? Did it introduce inappropriate content? Did it escalate when required? Did the safeguards actually operate?
Those are measurable questions.
This Is Becoming a Liability Question Too
Hartman makes another provocative argument: Chatbots should not necessarily receive the same legal treatment as traditional internet platforms because they are generating the content involved in these interactions rather than simply hosting material created by someone else.
Courts will ultimately determine how that distinction develops legally. For companies building AI, waiting for those decisions would be a poor risk strategy.
The safer assumption is that organizations will increasingly be held accountable for foreseeable behavior produced by the AI systems they deploy.
That makes behavioral governance part of product architecture, risk management, and eventually procurement. Enterprises will want to know which behaviors are technically prevented, which conditions trigger intervention, and what evidence exists that those controls actually work.
The standard for responsible conversational AI cannot simply be that the model usually behaves appropriately.
When AI is interacting with a child, “usually” is nowhere near good enough.
