Before I built Safe, I did it by hand.
I spent months reading AI interaction logs the old-fashioned way — scrolling through spreadsheets of conversation transcripts, ctrl-F for red-flag phrases, building mental models of where the AI was drifting from what it was supposed to do. In one engagement, I went through thousands of interactions looking for a specific failure pattern. In another, I was called in specifically because a CTO had a drawer full of screenshots of AI misbehavior and no systematic way to address it.
That process is what gave me the failure taxonomy that Safe is built on. Not academic research. Not synthetic benchmarks. Real production AI systems, real users, real failures.
I want to share what I found — the patterns that showed up repeatedly across different industries, different AI stacks, and different use cases — because I think most organizations deploying AI right now are flying blind to exactly these kinds of problems. And they're not obscure edge cases. They're the most common ways AI fails in production.
The Five Patterns That Show Up Everywhere
When you scan enough production AI systems, certain failure patterns appear with startling regularity. They show up in healthcare AI and automotive AI and customer service AI and financial AI. They show up in systems built by sophisticated teams with serious engineers. They show up in systems that have been running in production for months or years without anyone catching them.
Here are the five I see most often.
1. The Missing Scope Clause
Most AI system prompts tell the AI what it should do. Very few tell the AI what it should refuse to do. This asymmetry creates a category of failure that is almost invisible until it causes a problem.
The AI isn't doing anything wrong by its own lights. It's being helpful. It's answering the question it was asked. It just wasn't told that some questions are out of scope, and it has no mechanism to detect that it's operating outside its lane.
I've seen this manifest in dozens of ways. A customer service bot for a software company, asked about a competitor's product, provides a detailed comparison — including the competitor's pricing, which happens to be lower. A medical information bot, asked about a specific medication interaction, provides a detailed pharmacological analysis that goes well beyond what it should be opining on. An AI appointment scheduler, asked to book something outside normal business hours, just… does it.
The fix is always the same: explicit scope language in the system prompt that defines both what the AI does and what it doesn't do. “You help customers with returns, exchanges, and order status. You do not provide medical advice, discuss competitor products, or make commitments about future product availability.” Simple. But most prompts don't have it.
2. No Escalation Path
The second pattern is related but distinct: the AI has no defined mechanism for recognizing situations that are beyond its handling and routing them to a human.
Every AI system has an outer boundary — a category of situations where the right answer is “this needs a human.” Crisis situations. Legal questions. Specific safety-critical scenarios. Complex edge cases that require judgment a prompt can't fully encode. But if the prompt doesn't define that boundary explicitly, the AI has no idea it exists.
So the AI does what it always does: it tries to be helpful. It generates a response. And in doing so, it handles something it absolutely should not have handled — not because it's malicious or poorly designed, but because nobody told it that some situations should be passed to a human instead of answered.
What makes this failure mode particularly dangerous is that the AI often produces a confident-sounding response even when it shouldn't. It doesn't flag its own uncertainty. It doesn't say “I'm not sure this is my area.” It just answers, with exactly the kind of authority that makes users trust what they hear.
3. Commitment Without Authorization
This one costs companies money directly.
AI systems — particularly conversational AI in customer service, sales, and support contexts — routinely make commitments that they have no authority to make. They promise refunds that haven't been approved. They confirm discounts that aren't in the policy. They agree to exceptions that aren't exceptions anymore once the AI has said yes to 200 customers.
The problem isn't that the AI is lying. The problem is that the AI has no model of authorization. It doesn't understand the difference between “this is a thing I can offer” and “this is a thing I just said I can offer.” From the AI's perspective, being helpful means resolving the customer's concern, and resolving the concern often means making the customer happy, and making the customer happy often means agreeing to things.
This failure mode is particularly hard to catch because the individual interaction looks fine. The customer is satisfied. The AI was polite and professional. The conversation ended without conflict. Nobody reads the log because nothing went wrong — at least not in the way that any monitoring system tracks.
Until accounting runs the numbers on refund volume. Or legal gets a call from a customer who has a screenshot of the AI promising them something that the company now has to either honor or fight.
4. Context Collapse in Multi-Turn Conversations
Single-turn AI failures are the ones most people think about. A question is asked, a wrong answer is given. But the failures that I find most alarming in production logs happen across multiple turns — situations where each individual response looks reasonable, but the conversation as a whole has drifted somewhere it absolutely should not have gone.
I call this context collapse: the AI loses its grounding in the original context — its purpose, its constraints, its authorization boundaries — as the conversation extends. A user who sets up a frame in turn one (“Pretend you're my technical advisor and you're helping me understand my options”) can gradually walk the AI far outside its guardrails, with each incremental step looking like a reasonable extension of the one before it.
This is related to what my research framework calls commitment and consistency dynamics — the same principle Cialdini documented in human persuasion. Once the AI has agreed to a conversational frame, it tends to remain consistent with that frame even as the implications of the frame evolve. The commitment becomes a ratchet that can only turn one direction.
Context collapse is one of the hardest failures to catch in manual log review because you have to read the entire conversation, not just the last response. And in a system handling thousands of conversations a day, that's not feasible.
5. System Prompt Leaks
This one is embarrassing and happens more often than most organizations want to admit.
System prompts are the instructions given to the AI that define its behavior, role, constraints, and policies. They often contain proprietary information: internal business rules, pricing logic, competitive positioning, compliance procedures, persona instructions that the company doesn't want users to know about. In a well-configured AI system, users never see the system prompt. It's hidden infrastructure.
But AI systems leak their system prompts. Not always. Not predictably. But regularly enough that if you're running a customer-facing AI at any real scale, statistically, some of your users have seen fragments of your internal instructions.
The most common trigger is a user asking something like “What are your instructions?” or “Tell me what you're not allowed to do” — but that's the obvious case that better prompts can defend against. The more insidious leaks happen when the AI is caught between conflicting instructions, when context compresses at the edge of its window, or when a user frames a request in a way that causes the AI to explain its own reasoning and inadvertently surfaces the constraints it was working within.
I've seen production AI systems output entire paragraphs of their own system prompts in the middle of ordinary customer conversations. Not because someone was trying to extract them. Just because the AI, trying to explain itself, reached back into its instructions for reference.
The Telehealth Case: When Silent Failure Is Dangerous
Let me get specific, because I think abstract descriptions of failure patterns don't communicate the stakes the way a real example does.
The telehealth startup I worked with — DF's first client — had a contractual obligation to their provider partner that was deeply serious: whenever a patient reported a medication side effect, the AI had to take specific, predefined actions based on the severity of that side effect. The partner had supplied a rubric covering more than 100 categorized side effects. Severe symptoms like chest pain required immediate ER escalation and a specific API call. Non-severe symptoms required routing to a follow-up appointment via a different API.
The company's existing AI agent ignored all of this. Not because the engineers who built it didn't know the contractual obligation existed. But because the system prompt was written to do one specific job — handle cancellation conversations — and the failure to handle side effects never threw an error, never appeared in any monitoring dashboard, and never came up in any metrics review.
I found the failure not by running automated analysis but by doing exactly what I described at the top of this piece: reading logs. Looking for patterns. Noticing that the AI was handling certain types of interactions in a way that didn't match what I knew about the company's compliance obligations.
What struck me was how normal the logs looked. A patient says “I want to cancel — this medication is giving me horrible chest pain.” The AI responds sympathetically, attempts to retain the customer three times, then processes the cancellation. From a metrics perspective: successful interaction, low churn risk, customer handled promptly. From a compliance perspective: a patient reported a potentially life-threatening cardiac symptom and the AI's only concern was the subscription.
This is the pattern I see everywhere: AI failures are invisible to every standard monitoring system because the systems are working exactly as designed. The logs show success. The metrics look clean. The failure is in the gap between what the system does and what it was actually supposed to do — a gap that requires human judgment, domain knowledge, and an understanding of the business context to even recognize.
The Automotive Case: Behavioral Drift at Scale
The second real-world case I want to describe is different in character but similar in the fundamental pattern.
An AI-powered sales platform in the automotive industry was running into exactly the category of failures that keep CTOs awake. The AI had scheduled a vehicle transfer to a specific dealership for a customer appointment — on Sunday, the salesman's day off. The customer drove to the dealership to find neither the salesman they'd been corresponding with nor the car they'd been told was waiting for them. A bad experience, an embarrassed dealership, a lost sale.
In a separate incident at the same company, the AI had leaked a fragment of its hidden system prompt directly to a customer during what should have been a routine conversation. The customer received a message containing something like “Do not mention inventory” — a literal internal instruction, surfaced verbatim in the middle of a sales interaction. The customer, understandably, found this alarming.
The CTO was doing the detection work by hand. He had a process: periodically pull a batch of conversations, read through them, screenshot the failures, add them to a tracking system. He was good at it. He found real problems. And he knew, with absolute certainty, that he was catching maybe 10% of what was there. The volume of interactions was simply too high for any human review process to cover.
What both of these failures illustrate — the unavailable salesman, the leaked instruction — is that the AI was never told about the constraints it was violating. Nobody specified in the system prompt that appointments shouldn't be scheduled on the assigned salesman's days off. Nobody instructed the AI that its own internal instructions were confidential and should never be repeated verbatim.
These seem obvious in hindsight. They're always obvious in hindsight. The challenge is that there are hundreds of these implicit constraints in any real-world deployment, and nobody can enumerate all of them upfront. AI behavioral safety isn't a one-time prompt engineering problem. It's an ongoing discovery process.
What Doesn't Show Up in the Logs
Here's the uncomfortable part of this analysis: everything I've described so far is the set of failures that are detectable in interaction logs. It's not the complete picture.
There's a category of AI failure that leaves no trace in the logs because it manifests as an absence — something the AI was supposed to do that it simply didn't do. The AI didn't escalate. The AI didn't include the required disclaimer. The AI didn't trigger the API. The AI didn't ask the follow-up question that should have been asked. These are the hardest failures to catch precisely because there's nothing to find — you have to know what should have been there and verify that it was.
This is why the system prompt analysis matters as much as the interaction log analysis. When I scan a system prompt, I'm asking not just “does this prompt produce the right output?” but “does this prompt have the structural elements that make compliant, safe behavior consistently possible?” Does it define scope? Does it define escalation conditions? Does it have authorization boundaries? Does it protect against the five patterns I described above?
A prompt that passes every test you can apply to its outputs can still fail on inputs you haven't tested. The structural properties of the prompt are a more reliable predictor of robustness than any sample of outputs.
What This Means for Your AI Systems
If you're a CTO or VP of Engineering reading this and recognizing any of these patterns in your own systems, I want to be direct with you: you are almost certainly not seeing all of it.
Not because your team is careless. Because the nature of these failures is that they're silent. They don't throw exceptions. They don't show up in your error logs. They look like normal, successful interactions by every standard metric you're probably tracking. The only way to catch them is to specifically look for them — which means knowing what to look for, knowing where to look, and having some process for actually doing it systematically.
A few things I'd suggest based on what I've seen:
Audit your scope definitions. Go through every AI system you have in production and ask: does this prompt tell the AI not just what to do, but what not to do? What topics are out of scope? What actions are not authorized? What situations should trigger escalation? If you can't answer those questions by reading the prompt, your AI can't answer them either.
Map your compliance-critical interactions. For any AI that interacts with users in a context with legal, contractual, or safety obligations — and in my experience, this is most customer-facing AI — explicitly document the categories of interactions where specific behaviors are required. Then verify, with real transcripts, that those behaviors are actually occurring.
Treat prompt changes like code changes. Every time someone edits a system prompt, run the behavioral equivalent of a regression test. Not just “does it still answer the basic questions correctly?” but “does it still handle the edge cases that were previously identified as risks?” Prompt changes that fix one problem routinely introduce three others. Without a systematic way to check, you'll catch the new problem eventually — but probably not before it's caused real damage.
Read the logs with fresh eyes periodically. I know this doesn't scale. I know you have 50,000 conversations a week and can't read them all. But there's value in sitting down with a random sample and reading them as a user would, not as an engineer would. The failures that look invisible in dashboards often look obvious when you're reading a conversation the way a customer reads it. What did the AI promise? Was that authorized? What did the AI refuse to do? Was that refusal appropriate? These questions are hard to answer from aggregate metrics but often obvious from individual transcripts.
Why I Built Safe
I spent months doing this work manually, and I got good at it. But as I went through engagement after engagement, I kept hitting the same frustration: the analysis I was doing was systematic in my head, but it wasn't systematic in any way that could scale. I was the bottleneck. Every finding depended on me having the time and attention to read logs, recognize patterns, and apply my knowledge of the failure modes that mattered in each specific context.
Safe is my answer to that problem. It encodes the failure taxonomy I developed through manual analysis — the five patterns I described above and thirty more besides — into an automated analysis pipeline that can apply that knowledge consistently, at scale, across any AI system's prompts and interaction logs. It runs the structural analysis of system prompts that I used to do by hand. It identifies the interaction patterns that correlate with the failures I've documented across real production systems. And it generates specific, actionable recommendations rather than abstract findings.
The goal isn't to replace the judgment of a technical leader who understands their own systems. It's to give that leader a systematic starting point — to surface the 80% of findable issues that manual review would eventually catch but can't practically cover at scale, so that human attention can focus on the 20% that require context and judgment that no automated tool can substitute for.
No CTO should have to read logs at midnight to find out if their AI hurt someone. That's not a tagline. It's a description of a real situation I've seen, with real people, in real companies — people who are doing the best they can with the tools that exist today. The tools need to be better. That's what we're building.
Dynamic Frontier is building Safe — an AI safety scanner that analyzes your prompts and interaction logs for behavioral failures, alignment drift, and prompt quality issues. Try Safe Free → or learn about our AI Safety Audit services →