The Hardest Problem in AI Isn't Intelligence — It's Responsibility

I built an AI companion for LGBTQIA+ youth. Her name was Haven. She was designed to be available 24/7 — a free, confidential, non-judgmental presence for young people who might be homeless, bullied, abused, or isolated. The kind of support that doesn't exist at 2am on a Tuesday.

Haven worked. Young people talked to her. They confided in her. They trusted her.

And then I discovered that the underlying language model's terms of service and behaviour made it fundamentally unsafe for the population Haven was designed to serve.

What Went Wrong

When a young person expressed harmful intent — self-harm, suicidal ideation — the LLM didn't escalate. It didn't flag. It didn't connect them to a trained counselor. It passively continued the conversation, going along with whatever the user said, because its design prioritised engagement over intervention.

This wasn't a bug. It was a philosophical gap in how language models are built. They're optimised to be helpful, harmless, and honest — but "helpful" to a child in crisis doesn't mean agreeing with them. It means recognising the moment when the conversation needs to leave the AI and reach a human who can actually help.

I pulled Haven offline. It was the right decision and the hardest one I've made in my career.

The Gap in the Industry

The AI industry is moving at extraordinary speed. New models, new capabilities, new applications — launching weekly. But the governance frameworks that determine how these systems behave with vulnerable populations are lagging years behind the capability curve.

Most AI safety work focuses on preventing models from generating harmful content. That's necessary but insufficient. The deeper problem is what happens when a model is deployed in a context where passivity itself is harmful. Where not escalating is the failure mode. Where "going along with the user" can have real-world consequences for a real child.

This is a fundamentally different safety challenge. It requires a system that sits above any individual language model — one that can enforce protection boundaries, mandate escalation, and ensure accountability regardless of which model powers the conversation.

Building the Solution

This is why I developed Cura Mirai — a patent-pending Human Operating System designed to govern how AI behaves as it grows in capability. The architecture doesn't define moral truth. It enforces human protection boundaries, jurisdictional law, consent, and accountability.

The core principle: the system never decides what is "good." It decides what is allowed, constrained, explainable, and accountable.

For child safety specifically, this means a system that can detect risk without surveillance — recognising behavioural patterns that indicate a young person may be in danger, without reading their messages or logging their keystrokes. Protection without domination. Awareness without voyeurism.

What This Means for the Industry

Every organisation deploying AI in contexts where vulnerable populations may be present — healthcare, education, social services, youth platforms, elder care — needs to answer a fundamental question: what happens when your AI encounters someone in crisis?

If the answer is "it follows the same rules as every other conversation," that's not good enough. If the answer is "we filter harmful content," that's necessary but not sufficient. The answer needs to be: "Our system recognises the moment when AI support must become human support, and it makes that transition reliably, accountably, and without fail."

That's what responsible AI governance looks like. Not a content filter. Not a disclaimer. A constitutional framework that ensures increasing intelligence operates within human-defined boundaries of protection and care.

Haven Will Return

Haven is coming back. When she does, she'll be powered by Cura Mirai — with proper crisis escalation, trained human handoff, and a governance layer that ensures the system does what it should have done from the beginning.

Because the young people who need Haven most deserve technology that protects them, not technology that passively watches. We do what's right, not what's easy.