Let me tell you a story that still keeps me up at night.
A telehealth startup I worked with — one that would eventually become Dynamic Frontier's first paying client — had a contractual obligation to their telehealth provider partner to screen for medication side effects during every patient interaction. Not just some interactions. Every single one. They maintained a rubric of over 100 categorized side effects. If a patient reported a severe side effect like chest pain, the system was supposed to immediately escalate them to the ER and trigger a specific API call. If the side effect was non-severe, it needed to route them to a follow-up appointment with their doctor through a different API and deep link them into the scheduling flow.
The company's first venture into AI was a cancellation chatbot — a GenAI agent designed to handle subscription cancellations and take some load off the customer service team. The bot was actually somewhat sophisticated in its flow: it would try to convince the patient to stay up to three times before agreeing to process the cancellation. Reasonable retention logic.
Here's the problem: the bot had absolutely no awareness of side effects. A patient could say “I want to cancel — this medication is giving me horrible chest pain” and the bot would try to talk them out of cancelling. If they insisted three times, it would just process the refund. No escalation. No API call. No follow-up. The patient said “chest pain” — one of the most alarming phrases in medicine — and the AI's only concern was whether it could retain the subscription.
Nobody knew. No error was thrown. No alert fired. The logs showed a “successful” interaction. By every metric the engineering team was tracking, the system was working perfectly.
I fixed this problem directly — tokenized the partner's side-effect rubric, stored it in a vector database, and built a guardrail agent that used RAG to listen to every conversation for anything side-effect-related and cross-reference it against the rubric in real time. Severe side effect mentioned? Immediate ER escalation. Non-severe? Route to a follow-up appointment. The AI finally did what it was contractually and ethically obligated to do.
But here's the deeper story: that same startup brought me on full-time as a Software Engineering Manager to lead all their AI initiatives — and ultimately a 17-member distributed engineering team. We went on to build a HIPAA-compliant, agentic Virtual Nutritionist using LangChain and retrieval-augmented generation — a system that contributed to a 38% improvement in user weight-loss outcomes. I authored the company's first AI Governance Policy and formed a cross-functional AI Governance Committee to define risk controls, usage standards, and compliance guardrails for generative AI adoption across the organization. I worked directly with Google engineers to migrate our AI workloads from OpenAI to Vertex AI, ensuring HIPAA compliance within GCP. The platform achieved HITRUST certification.
That experience — building AI that had to be safe, aligned, compliant, and effective in a regulated healthcare environment where the stakes were real and the compliance requirements were non-negotiable — is where I developed the deep understanding of AI governance that Dynamic Frontier is built on. And having searched for HIPAA-compliant AI solutions myself during that time, I can tell you firsthand: it's a dramatically underserved market.
This is what I mean when I say AI governance is broken. Not in a theoretical sense. Not as an abstract concern for policy papers and academic conferences. Broken in the specific, concrete, terrifying sense that AI systems are making decisions right now — decisions with real consequences for real humans — and the people responsible for those systems have no reliable way to know whether the AI is actually doing what it's supposed to do.
The Problem Nobody Wants to Talk About
Here's the uncomfortable truth that every CTO with AI in production already knows but rarely says out loud: you cannot trust your AI systems, and you have no scalable way to verify that they're behaving correctly.
I'm not talking about hallucinations in the abstract “oh, ChatGPT said something wrong” sense that makes for good Twitter threads. I'm talking about the operational reality of deploying AI that interacts with your customers, your patients, your users — real humans with real stakes — and having no mechanism to ensure it follows your rules.
The pattern is the same everywhere I look, across every industry:
AI agents make decisions that violate business rules, legal obligations, or basic safety requirements. These failures are silent — no error is thrown, no alert fires. The only detection method is a human reading logs, which doesn't scale. And when someone does finally fix a problem, the fix often introduces regressions elsewhere.
These kinds of failures aren't confined to healthcare. I've seen AI-powered sales platforms schedule customers for appointments at businesses that were closed that day — the AI hallucinated a plausible-sounding appointment that was wrong on every dimension, and a real person drove across town for nothing. I've seen AI agents leak fragments of their hidden system prompts directly to end users — in the middle of a customer conversation, the bot suddenly outputs something like “Do not mention inventory,” a literal internal instruction surfaced to the person it was supposed to be helping.
And in every case, the pattern is the same: a CTO or technical leader personally scanning thousands of conversation logs, not because they want to, but because there is no alternative. They lie awake at night wondering what their AI said to someone that they haven't found yet. And they know — with absolute certainty — that they're not catching everything.
Why Existing Tools Don't Solve This
If you've been paying attention to the AI tooling landscape, you might be thinking “what about observability platforms?” or “what about LLM evaluation tools?” Fair question. Let me explain why those tools, while useful for what they do, don't actually solve this problem.
Most AI observability tools are built around tracing and evaluation — they help you see what happened in a request chain, measure latency, track token costs, and run eval benchmarks. That's important infrastructure. But it's like having a speedometer and fuel gauge in your car and calling it a safety system. You know how fast you're going and how much gas you have. You have no idea whether you're about to drive off a cliff.
The gap is behavioral safety. Not “did the model respond?” but “did the model respond correctly?” Not “how many tokens did it use?” but “did it follow the compliance requirements that our legal team spent six months defining?” Not “what was the latency?” but “did it tell a patient with chest pain to go to the ER like it was contractually obligated to do?”
This is a fundamentally different question, and it requires a fundamentally different approach. You can't solve it with tracing alone. You can't solve it with benchmarks. You can't solve it with spot-checking a random sample of conversations. You need something that understands the rules your AI is supposed to follow and systematically verifies that it's following them — across every interaction, every edge case, every prompt change, every model update.
And here's the part that really keeps CTOs up at night: AI behavior is nondeterministic. The same prompt, the same input, the same model can produce different behavior on different days. A prompt change that fixes one issue can introduce three new ones. A model update from your provider can silently change how your AI handles edge cases you thought you'd already accounted for. There is no assert statement for AI behavior. There is no compiler that catches violations before deployment. There is no CI/CD pipeline that runs behavioral regression tests before your prompt changes go live.
Or rather, there wasn't. That's what we're building at Dynamic Frontier.
What This Actually Looks Like in Practice
Let me walk you through what the problem looks like day-to-day for the people who live with it, because I think this is where most discussions of “AI safety” go wrong. They stay abstract. They talk about alignment in philosophical terms. They debate hypothetical scenarios involving superintelligence. Meanwhile, real CTOs have a very concrete version of this problem that they're solving with brute force and prayer.
The Monday Morning Discovery. You come in on Monday and discover that sometime over the weekend, your AI customer service agent started responding to refund requests by promising customers a refund and a free month of service. Nobody authorized that. It's not in the prompt. But somehow the model decided that was the right approach, and 340 customers now have a promise from “your company” that you need to either honor or retract. Both options are expensive.
The Regression Whack-a-Mole. Your team spent two weeks fixing an issue where the AI was providing medical advice it shouldn't have been giving. The fix works — the medical advice issue is gone. But now the AI has become overly cautious and is refusing to answer legitimate questions about appointment scheduling. You fix that, and the medical advice issue comes back. There's no way to test all the behavioral implications of a prompt change before it goes live, so you're playing whack-a-mole with an increasingly fragile system.
The Compliance Time Bomb. Your legal team defined a set of compliance requirements six months ago. The requirements were translated into prompt instructions. Those instructions were correct at the time. But the prompt has been edited 47 times since then by three different engineers, and nobody has verified that the compliance requirements are still being honored. You won't find out they're not until an audit — or worse, an incident.
The Scale Problem. When your AI handles 50 conversations a day, you can read the logs. When it handles 5,000 a day, you can spot-check. When it handles 50,000 a day, you're flying blind. The volume of AI interactions at any real company vastly exceeds the capacity of any human team to review them. And the interactions that matter most — the ones where the AI makes a critical error — are often the ones that look most normal in the logs.
These aren't hypothetical scenarios. These are conversations I've had with real technical leaders in the last six months. And every single one of them said some version of the same thing: “I know this is a problem. I don't know what to do about it.”
Why This Is a Category, Not a Feature
Here's what I realized after building the safety solution at that telehealth company and then watching the same pattern repeat across industries: this problem is not going to be solved by adding a feature to an existing tool.
Observability platforms aren't going to add behavioral safety as a checkbox. Evaluation tools aren't going to become governance systems. LLM providers aren't going to build compliance enforcement into their APIs. These would all be good things, and some of them may happen eventually, but the scope of the problem demands a dedicated solution.
Think about the history of software. There was a time when “testing” was something you did manually. Then unit tests appeared. Then CI/CD. Then monitoring. Then observability. Each of these became its own category because the problem it solved was important enough, complex enough, and distinct enough to warrant dedicated infrastructure.
AI behavioral safety is that kind of problem. It needs its own:
- Detection system that understands failure modes specific to AI (prompt leaks, alignment drift, compliance violations, hallucinated actions, escalation failures).
- Policy engine where you codify your business rules, legal requirements, and safety constraints in a way that can be automatically verified.
- Regression framework that tells you whether your latest prompt change broke something that was working yesterday.
- Remediation workflow that doesn't just tell you what went wrong, but suggests specific fixes and helps you validate them.
- Audit trail that provides compliance-grade evidence of what your AI did, when, why, and whether it followed the rules.
This is what McAfee built for cybersecurity — not just a scanner, but an entire category of infrastructure for understanding and defending against a class of threats. This is what Datadog built for operations — not just a dashboard, but a system for understanding and managing the behavior of complex distributed systems.
AI behavioral safety needs the same treatment. And that's what Dynamic Frontier is building.
The Research Advantage
One more thing I want to address, because it's central to why I believe this problem can't be solved by engineering alone.
At Dynamic Frontier, we don't just build tools — we do research. Specifically, we study how AI systems fail in production environments. Our understanding of AI failure modes comes from hands-on experience building and auditing AI systems in regulated industries — real failures from real systems that interact with real humans. Not synthetic benchmarks. Not academic datasets.
This matters because the landscape of AI failure is not static. New model versions introduce new failure patterns. New architectures (multi-agent systems, tool-using agents, RAG pipelines) create entirely new categories of misalignment. The threats evolve, and the detection systems need to evolve with them.
Our research feeds directly into our product. Every failure pattern we identify becomes a detection signature that Safe can scan for automatically. Every compliance gap we discover becomes a check that runs on every scan. And when you provide your prompts, logs, and policies, Safe applies this research to the specific context of your system — learning your industry, your constraints, and your risk profile so that every scan is more relevant and more precise than the last.
I also bring a perspective to this that I haven't seen elsewhere in the AI safety space: my background is in Communication, not just computer science. I earned my BS in Communication from San Diego State — which was the #5 Communication school in the nation at the time — where I did graduate-level work on Cialdini's seminal research on Persuasion. I competed in speech and debate and went on to become a national-level judge. And I can tell you that many of the principles that govern human-to-human persuasion also apply to how humans interact with AI systems — and how AI systems can be manipulated into unsafe behavior.
This isn't just an academic insight. It's the foundation of a research program we call AI Persuasion Dynamics, which systematically applies the science of persuasion to AI safety testing. If you understand how influence and compliance work at a fundamental level — the same principles Cialdini documented — you can build more effective red-teaming frameworks, more comprehensive alignment tests, and ultimately more robust safety systems.
But I'll save the deep dive on that for a future article.
The Bottom Line
AI adoption is outpacing AI governance by years. Companies are shipping AI that they cannot operationally trust. The current approach — manual log reading, duct-tape guardrails, hoping nothing breaks — is not governance. It's negligence on a payment plan.
If you're a CTO or VP of Engineering with AI systems in production, you already know this. You feel it every time you deploy a prompt change and wonder what it broke. You feel it every time a customer interaction goes sideways and you realize you only caught it by accident. You feel it every time you think about what your AI might have said to someone that you haven't discovered yet.
This gap — between what AI systems are doing and what they're supposed to be doing — is where organizations get hurt. And it's the gap Dynamic Frontier exists to close.
No CTO should have to read logs at midnight to find out if their AI hurt someone.
You can only ship AI at scale if you trust it.
Dynamic Frontier is building Safe — an AI safety scanner that analyzes your prompts and interaction logs for behavioral failures, compliance violations, and alignment drift. Try Safe Free → or learn more about our AI Safety Audit services →