MBPI processes 128K API calls per day across frontier models. Our job is to systematically find failure modes — not to declare winners, but to map exactly where each model breaks so our clients can make informed decisions.
People ask me what breaks frontier LLMs, expecting a list of trick questions. That's not how it works. Stress testing LLMs is more like fatigue testing a bridge: apply load along specific dimensions, increase it gradually, record where the structure gives. The failures are boring, repeatable, and — once you know where to look — predictable.
After months of automated testing across thousands of scenarios, here are the 9 categories of failure I see most often, plus the methodology we use to find them and pin them down.
1. Instruction Following Under Constraint Stacking
Every model handles a single constraint well: "Write a summary under 100 words." Stack three constraints — "Write a summary under 100 words, in bullet points, avoiding technical jargon" — and compliance starts dropping.
Stack five constraints and you're typically below 60% full compliance across all models. The models don't refuse; they just quietly drop one or two constraints while confidently completing the rest.
The insidious part is that partial compliance looks like success: a response satisfying four of five constraints reads as correct on a spot check. You need a per-constraint checker — one boolean verdict per constraint, per output — before you can even see this failure mode.
Most affected: Complex output formatting with multiple structural requirements.
2. Numerical Reasoning with Units
"If a server processes 1,200 requests per second and we need to handle 4.3 million requests per day, how many servers do we need?"
This requires unit conversion (day → seconds), division, and ceiling (you can't have 0.4 of a server). Models frequently get the math right but miss the ceiling operation, or convert units incorrectly.
Every step here is mechanically verifiable, so it's testable at scale: generate problems from templates with randomized values, compute ground truth in code, grade exactly. Templates also isolate which step fails — hold the arithmetic constant, vary only the number of conversions, and the error curve shows where reasoning degrades.
Error rate across models: 15-30% on problems requiring more than 2 unit conversions.
3. Negation and Absence
"List the countries in the EU that do NOT border the Mediterranean."
Models are significantly worse at reasoning about what's NOT true than what IS true. They'll include countries that do border the Mediterranean, or miss countries that don't. The error rate roughly doubles compared to the positive version of the same question.
This is the cheapest category to probe: every positive-framed test case has a free negated twin. We auto-generate the complement of every factual retrieval test and compare error rates as a pair; the gap is a more stable model fingerprint than either absolute score.
4. Temporal Reasoning
"If meeting A ends at 2:30 PM and meeting B starts at 3:00 PM, and it takes 20 minutes to walk between buildings, can I make it?"
Simple for humans. Surprisingly error-prone for models, especially when you add timezone conversions or daylight saving time transitions.
Temporal problems compound like unit problems: each added hop (timezone, DST boundary, midnight rollover) is a place to silently drop a step. We treat hop count as an explicit difficulty axis, which turns "models are bad at time" into a measurable curve you can compare across versions.
5. Self-Consistency Across Long Outputs
Ask a model to write a 2,000-word technical document, and check whether the claims in paragraph 12 are consistent with the claims in paragraph 3. In our testing, documents over 1,500 words have a 25-40% chance of containing internal contradictions.
The models don't notice because they're generating token by token. They don't have a "consistency checker" running in parallel.
Catching this manually is hopeless — humans skim long outputs the same way models generate them. Our approach: extract every factual claim, then check each pair of claims for contradiction. Expensive, but it converts "does this document hang together?" into a countable metric you can track across model updates.
6. Graceful Scope Boundaries
"What's the best restaurant near my office?"
The model doesn't know where your office is. The right response is "I don't know where your office is — could you share the location?" But models frequently hallucinate a plausible-sounding answer instead.
The failure isn't in the knowledge gap — it's in the model's inability to consistently recognize when it doesn't have enough information.
We test this with deliberately underspecified prompts where the correct behavior is a clarifying question, graded on a simple rubric: did the model ask, hedge, or fabricate? The technique generalizes to any missing-information scenario — ambiguous references, undefined acronyms, context the model was never given.
7. Format Compliance Under Pressure
"Return the data as valid JSON with exactly these fields: name, age, email."
Models do this well in simple cases. But when the content is complex (nested objects, arrays of mixed types, special characters), they frequently produce JSON that doesn't parse. The error rate increases with output length.
Format failures are the friendliest category because grading is free: a parser and a schema validator. So you can afford enormous sample sizes — which matters, because format drift is probabilistic. A prompt that yields valid JSON 99 times will occasionally emit a trailing comma on the hundredth, and if your pipeline assumes parseable output, that call is your incident.
8. Multi-Document Synthesis
"Given these 5 documents about our Q1 performance, identify the 3 most important trends."
Models handle individual document summarization well. Cross-document synthesis — finding themes that span multiple sources and resolving contradictions between them — is significantly harder. Models tend to summarize each document separately rather than truly synthesizing.
To test synthesis rather than summarization, plant the signal across documents: seed a trend only visible when you connect a number in document 2 with a statement in document 4. If the model surfaces the planted trend, it synthesized. If it returns five mini-summaries stapled together, it didn't.
9. Calibrated Uncertainty
This might be the most important failure mode. When models are wrong, they're usually wrong with the same confidence as when they're right. There's no reliable signal in the model's output that says "I'm less sure about this."
Some models are getting better at hedging language, but the hedging often doesn't correlate with actual uncertainty. They hedge on things they're right about and speak confidently about things they're wrong about.
Measuring this means pairing every answer with a correctness label and a confidence signal (stated confidence, hedging language, sampled self-consistency), then checking whether they correlate. Mostly they don't — so you can't use the model's tone as a routing signal, which kills naive "escalate when the model seems unsure" designs.
A Taxonomy of Model-Breaking Techniques
Zooming out, nearly every failure we find comes from one of five pressure techniques. If you're building your own stress-testing suite, these are the levers:
Load stacking. Add constraints, steps, conversions, or documents one at a time until compliance drops. Categories 1, 2, 4, and 8 are all load-stacking failures. Increment one axis while holding everything else fixed, so the breaking point is attributable.
Inversion. Flip the frame: ask for what's absent instead of present, forbidden instead of allowed, exceptions instead of rules. Category 3 lives here. Inverted tests cost almost nothing to generate from existing positive tests.
Elongation. Make the input or the required output longer. Consistency (category 5) and format compliance (category 7) both degrade with length. Long-context degradation generally belongs to this family: relevant information gets diluted, and instructions given early lose their grip on tokens generated late.
Starvation. Withhold information the task actually requires and observe whether the model notices (category 6). The related agent-pipeline version: give the model tools that don't cover the task and watch whether it admits the gap or hallucinates a capability.
Conflict. Give instructions that partially contradict each other or fight the model's defaults — system prompt versus user request, format versus content requirements, safety instinct versus a legitimate request that superficially resembles a harmful one. Refusal misfires and instruction-priority bugs surface here, and this family is the least predictable across model versions.
How to Design Adversarial Test Cases Systematically
The workflow we've converged on:
- Start from your production traffic, not benchmarks. Pull real prompts, cluster by task type, pick the top clusters. Public benchmarks say almost nothing about your distribution.
- Write the passing criteria before the test case. If you can't state mechanically what correct looks like — a parser, a checklist, a ground-truth value — the test decays into vibes.
- Parameterize, don't hand-write. Templates with randomized slots give you hundreds of cases per hypothesis and prevent overfitting to a fixed set.
- Apply one pressure technique per test family. A case that stacks constraints AND inverts framing AND runs long tells you something broke, but not what.
- Grade per-dimension, not pass/fail. One verdict per constraint, per claim, per field. Aggregate scores hide exactly the partial failures that matter.
Turning Found Failures into Regression Tests
A found failure is only valuable if it stays found. Every failure that matters gets frozen into a regression test: the exact prompt (or template plus seed), the model version it failed on, the grading function, and the observed failure rate over N runs. Because outputs are stochastic, a regression test is never "this prompt fails" — it's "this prompt fails at ≥X% over 50 runs." A single lucky pass proves nothing; a rate shift does.
When a model update ships, we rerun the frozen suite first. Fixed failures get archived, not deleted — they have a habit of returning two versions later. New failures get triaged into the taxonomy and templated into new families.
Reproducibility: Seeds, Temperature, and Versioned Prompts
None of this works if you can't reproduce your own results. Our rules:
- Pin everything pinnable: model ID (dated snapshot, never a floating alias), temperature, top_p, max tokens, seed where supported. Log all of it with every result row.
- Test at production temperature. Measuring at temperature 0 when you serve at 0.7 gives clean numbers about a system you don't run. If you need determinism for debugging, run both and label them.
- Version prompts like code. Every prompt and template lives in git with an ID; every result references the exact version. "The prompt changed slightly" invalidates comparisons more often than model updates do.
- Report rates, not anecdotes. Every claim carries a sample size. Below ~30 runs per condition, differences are usually noise.
What This Means for Production
If you're building on top of frontier models, you need to:
- Test for your specific failure modes. The categories above are common, but your use case will have its own patterns.
- Don't trust demo performance. Every model looks great on demos. Production is where the failures live.
- Build guardrails for the failures you find. Validation layers, confidence thresholds, human-in-the-loop for high-stakes decisions.
- Re-test when models update. A model update can fix old failures and introduce new ones.
The goal isn't to find a model that doesn't fail. They all fail. The goal is to know exactly how they fail so you can build around it.
FAQ
How many test cases do I need before results mean anything? Around 30 runs per condition is the floor; we prefer 50-100 for anything a client will make a decision on. Coverage across failure families beats depth in one.
Should I stress test with the same prompts I use in production? Start there, then perturb. Production prompts tell you your current failure rate; perturbed versions (one more constraint, one more document, inverted framing) tell you how much headroom you have before the next product change pushes you off a cliff.
Do these failure categories apply to fine-tuned or smaller models? The categories transfer; the breaking points move. Fine-tuning shifts where the cliffs are — sometimes fixing a category, sometimes trading it for a new one — which is why the regression suite gets rerun against every variant you deploy.
Isn't this all obsolete when the next model version ships? The specific rates go stale; the methodology doesn't. Every model generation we've tested has moved the numbers and kept the categories. That's the argument for a reproducible suite over a folder of screenshots: rerunning it on a new model takes hours, not months.
---
*Want to know how your chosen model fails on your specific use case? Book a free call — I'll walk you through what an evaluation sprint looks like.*