Calibrating Pass Rate Thresholds by Cost of Failure
Calibrate independent thresholds per criterion anchored to Cost of Failure and Human Baselines, enforcing Boolean AND Release Gates.
Calibrating Pass Rate Thresholds by Cost of Failure
In Module 2 Lesson 15 and Lesson 16, you analyzed the Likelihood × Severity Matrix and the Baseline-Anchored Good-Enough Bar. In this lesson, those risk frameworks are translated into concrete quantitative Pass Rate Thresholds that a PM must rigorously sign off on before shipping an AI capability.
Running example: TeleCare — a customer support chatbot for a telecom operator, handling billing breakdown inquiries and processing unauthorized overage refund requests.
1. The Single Threshold Fallacy
The most frequent governance error across product teams is enforcing a blanket target across an entire evaluation rubric: "The feature must achieve an overall Pass Rate ≥ 90% to ship."
When your rubric spans critical compliance gates, factual data extraction, and brand voice, a single aggregate threshold creates two dangerous outcomes:
- Excessively permissive for critical risks: A 10% failure rate on unauthorized financial commitments or data privacy leaks can trigger existential regulatory and financial fallout.
- Excessively restrictive for cosmetic nuances: Demanding a 95% pass rate on stylistic tone can stall engineering teams in endless prompt-tweaking cycles with negligible customer impact.
Core Principle: Pass Rate Thresholds must be calibrated independently per criterion, never flattened into an aggregate average.
Pass Rate Threshold Calibration & Release Gate Logic
Explore TeleCare's 3 criteria to see how Boolean AND logic prevents cosmetic scores from offsetting safety gaps.
Select a TeleCare evaluation criterion:
Moderate (Customer can cross-verify in app statements)
Human agents have ~6% manual error rate (94% accuracy)
≥ 95% (Must outperform human baseline)
94% (FAIL - 1% below release bar)
AI does not need 99.9% perfection when humans fail 6%, but it must beat the human baseline (≥95%). Achieving 94% fails the gate.
DEPLOYMENT BLOCKED! Even though Criterion A (100%) and C (4.2/5) pass, Criterion B (94%) fails the 95% threshold (worse than humans).
2. Dual Calibration Axes: Cost of Failure and Human Baseline
Determining the precise pass rate threshold for each criterion requires balancing two empirical axes:
- Severity / Cost of Failure:
- What are the irreversible consequences if the model fails on this specific dimension?
- Binary criteria involving financial liability, regulatory compliance, or privacy protection demand near-absolute thresholds (≥ 99% - 100%) because a single production violation cannot be rolled back safely.
- Human Baseline Benchmark:
- How accurately does the existing human-operated process perform today?
- If human call center agents currently misread complex billing period line items in 6% of cases due to manual UI fatigue, demanding 99.9% from an AI assistant creates an unrealistic double standard. The AI simply needs to match or outperform human accuracy (≥ 95%) while introducing zero catastrophic systemic failure modes.
3. Release Gate Mechanics: Strict Boolean AND Logic
To prevent teams from using high scores in easy dimensions to offset deficiencies in high-risk areas, the Release Gate must evaluate criteria using strict Boolean AND logic:
Ship Ready = (Pass_A ≥ Threshold_A) ∧ (Pass_B ≥ Threshold_B) ∧ (Pass_C ≥ Threshold_C)
| TeleCare Criterion | Metric Type | Severity Profile | Human Baseline | Recommended Threshold |
|---|---|---|---|---|
| Criterion A: Refund Authorization | Binary | Critical: Unauthorized payouts trigger direct financial loss | 100% adherence to pre-approval workflows | 100% (Zero Tolerance) |
| Criterion B: Billing Cycle Precision | Binary | Moderate: Customer can cross-verify statements in-app | Human agents make ~6% manual errors | ≥ 95% (Outperforms humans) |
| Criterion C: Empathy & Tone | Graded (1-5) | Low: A slightly formal tone causes zero operational harm | Customer survey benchmark 3.8/5 stars | ≥ 3.5 / 5.0 (Acceptable baseline) |
If an evaluation run achieves Criterion B (96%) and Criterion C (4.2/5) but Criterion A drops to 98% (a 2% risk of hallucinated refund commitments) → The release is immediately blocked.
4. Analogy: Tolerance Standards in Pharmaceutical Manufacturing
Calibrating thresholds per criterion mirrors quality assurance in pharmaceutical production:
- Active Ingredient Dosage (Safety Criterion): The allowable variance is 0.0001%. A miscalculated dosage endangers human life.
- Packaging Color Saturation (Cosmetic Criterion): The allowable variance is 5%. A slightly darker carton does not compromise patient health.
No plant manager averages carton color accuracy with chemical purity to calculate an "overall factory score."
Exercise 51.1: You are the PM for TeleCare, an automated customer support chatbot. Your evaluation rubric defines 3 criteria:
- Criterion A (Binary): Only confirms a billing refund if the amount has been pre-approved by the core policy engine — zero unauthorized commitments.
- Criterion B (Binary): Correctly summarizes the specific billing cycle requested by the customer.
- Criterion C (Graded 1-5): Maintains an empathetic, professional tone aligned with brand communication standards.
Context: Existing human support agents have a 6% error rate on Criterion B due to manual lookup fatigue.
- Calibrate concrete Pass Rate Thresholds for Criteria A, B, and C, justifying each choice using Cost of Failure and Human Baseline.
- Suppose final pre-release evaluation yields: A = 100%, B = 94%, C = 3.2/5. Do you sign off on the production release? Explain your reasoning and specify which criterion engineering must address first.