AI Evaluation
Independent
Ongoing

MBPI: Model-Breaking Platform at 128K API Calls/Day

Building an automated platform to systematically find failure modes across frontier LLMs

128K calls/day
API Throughput
100K+/day
Automated Evaluations
0.89 Spearman vs expert labels
Judge Correlation
9 systematic categories
Failure Categories

The Challenge

AI teams choose models based on public benchmarks and demo performance. But public benchmarks test generic capabilities, and demos test the happy path. Neither tells you how a model will fail on your specific workload. There was no accessible platform for systematically stress-testing frontier models at scale across custom scenarios.

The Approach

  • Designed and solo-built the MBPI (Model Benchmarking & Performance Intelligence) platform from scratch
  • Architected for high throughput: handles up to 128K API calls/day across multiple frontier model providers
  • Built an LLM-as-a-Judge evaluation system processing 100K+ automated evaluations daily with 0.89 Spearman correlation to expert human labels
  • Implemented multi-judge consensus (3 model families), bidirectional comparison, and rubric decomposition for reliable automated scoring
  • Created failure taxonomy covering 9 categories: instruction following, numerical reasoning, negation, temporal reasoning, self-consistency, scope boundaries, format compliance, multi-document synthesis, and calibrated uncertainty

Tech Stack

Python
Multiple LLM APIs (OpenAI, Anthropic, Google, Tencent)
Custom orchestration
Statistical analysis
LLM-as-a-Judge

The Problem

Every AI team faces the same question: "Which model should we use?" The typical answer is based on public leaderboards and a handful of demo prompts. But public benchmarks measure generic capabilities, and demo prompts test the center of the distribution.

Production traffic lives in the tails. The model that scores highest on MMLU might hallucinate on your specific domain. The model with the best HumanEval score might fail at multi-step reasoning in your workflow. You can't know until you test on your actual use cases at scale.

What I Built

MBPI is an automated platform for systematically finding failure modes across frontier models. I designed and built it solo.

Architecture

The platform orchestrates API calls across multiple model providers — OpenAI (GPT-5.1), Anthropic (Claude 4.5), Google (Gemini 3 Pro), Tencent (Hunyuan), and others. At peak, it processes 128K API calls per day, running structured test scenarios and collecting responses for automated evaluation.

LLM-as-a-Judge System

Human evaluation doesn't scale. At 128K calls/day, you'd need an army of evaluators. So I built an LLM-as-a-Judge system that processes 100K+ evaluations daily.

Getting automated judges to be reliable is hard. Naive implementations have well-known biases: position bias (preferring the first response), verbosity bias (preferring longer responses), and self-preference bias (GPT judges prefer GPT outputs). My system addresses all of these:

  • Multi-judge consensus: 3 different model families evaluate each output
  • Bidirectional comparison: Every pairwise judgment is run in both orders
  • Rubric decomposition: Instead of one overall score, I score 4-6 specific dimensions
  • Calibration set: 200 human-labeled examples verify judge accuracy weekly

The result: 0.89 Spearman correlation with expert human labels at roughly 1/100th the cost of full human evaluation.

Failure Taxonomy

After months of systematic testing, I've identified 9 categories where frontier models consistently fail:

  1. Instruction following under constraint stacking — Stack 5+ constraints and compliance drops below 60%
  2. Numerical reasoning with unit conversions — 15-30% error rate on multi-conversion problems
  3. Negation and absence — Error rates roughly double compared to positive versions of the same question
  4. Temporal reasoning — Especially with timezone conversions and DST transitions
  5. Self-consistency in long outputs — 25-40% of documents over 1,500 words contain internal contradictions
  6. Graceful scope boundaries — Models hallucinate instead of admitting they lack information
  7. Format compliance under pressure — JSON validity degrades with output length
  8. Multi-document synthesis — Models summarize individually instead of synthesizing across sources
  9. Calibrated uncertainty — Confidence doesn't correlate with actual accuracy

Why This Matters

When I run an Evaluation Sprint for a client, this infrastructure is what powers it. Instead of testing with 20 hand-picked prompts, I can test with thousands of structured scenarios and get reliable automated scores.

The output isn't "Model X is better than Model Y." It's "Model X fails at constraint stacking and temporal reasoning on your specific scenarios, while Model Y fails at format compliance and scope boundaries. Here's which one is better for your use case, and here are the guardrails you need for either."

That's the difference between evaluation and benchmarking-theater.

Want results like these?

Book a free call. I'll listen to what you're building and tell you honestly whether I can help.