Rule-Based Routing: Deterministic Gates Before the Model Runs

Every technique covered so far in this series - Model Cascading, the Classifier-Based Router, Semantic Routing - decides where a request goes by evaluating it, whether that evaluation runs a model, a check, or a vector comparison. Rule-Based Routing is the one technique in the series that sometimes decides without evaluating anything at all.

A rule is an explicit, hand-written condition: a request header equals a known value, a user's account tier matches a known category, a keyword or regex pattern appears in the text. When a rule matches, the routing decision is made in native application code, before any model - cheap or expensive - ever sees the request. The Classifier-Based Router article already introduced this idea in miniature, calling it a hard override for asymmetric risk. This article makes the case that rules deserve a place at the front of the entire routing stack, not just as a safety net bolted onto one technique.

What Counts as a Rule

Rules split into two families, and the distinction matters for how much you should trust them.

Metadata rules match on structured, application-controlled fields: a request header, an authenticated user's subscription tier, a feature flag, an API endpoint the request arrived through. These are close to infallible - the application itself set the value, so there's no ambiguity to misjudge.

Content rules match on the text of the request itself, usually via regex or keyword lists: "refund" routes to a billing handler, "urgent" combined with certain domain terms triggers a hard override, a specific phrase format indicates a known intent. Content rules are strictly weaker than metadata rules, because natural language is the one input a fixed pattern can't fully anticipate. A user will eventually phrase a routine request in a way that happens to contain your trigger keyword, and a rule has no way to notice that anything went wrong.

{
  "rule_id": "RULE_META_01",
  "matched": true,
  "match_type": "Exact Metadata Header Match"
}

Why Rules Come First in the Stack

The economic argument for putting a rule engine in front of every other routing technique is almost embarrassingly simple: a matched rule costs nothing and takes about a millisecond. No LLM call, no embedding computation, no verification pass. If a meaningful share of your traffic can be classified with total confidence from metadata alone - a mobile app's internal telemetry requests, a known integration partner's API calls, a user who selected "billing" from a dropdown before typing anything - routing that share through a classifier or a semantic router is pure waste.

This is also why rules compose naturally with every other technique instead of competing with them. A rule engine isn't an alternative to a Classifier Router or Semantic Routing - it's a filter that sits in front of either, absorbing the traffic it can classify with certainty and handing off everything else unchanged.

The Brittleness Problem: No Confidence Score to Catch a Miss

A rule's greatest strength - it never hesitates - is also its structural weakness. A classifier that misjudges a request still emits a confidence score; a human reviewing production telemetry can spot the low-confidence calls and investigate. A rule has no concept of "I'm not sure." It either matches or it doesn't, and when it matches for the wrong reason, nothing in the system signals that anything unusual happened.

This produces two distinct failure modes. A false negative - a rule that should have matched but didn't, because the user phrased things slightly differently than the pattern anticipated - simply falls through to the next tier, which is a graceful failure with no real cost beyond a marginally higher triage bill. A false positive - a rule that matches text it was never meant to match - is the dangerous one. A keyword written for a medical emergency override firing on an unrelated sentence that happens to contain the same word sends a routine request down an expensive, and sometimes inappropriate, path with no self-correction mechanism anywhere in the pipeline.

Rule-Based Routing vs. Classifier Routing

The Cost That Never Hits the Token Bill

Every other technique in this series has a cost you can read off an API invoice. Rule-based routing's real cost lives somewhere else entirely: the engineering hours spent writing, testing, and updating the rule table as the product surface grows. A rule list that covered every clean category at launch quietly accumulates blind spots as new features ship, new user segments appear, and language drifts - and unlike a classifier, which adapts the moment you edit its prompt, a rule table only gets safer when someone notices the gap and closes it by hand.

This isn't a reason to avoid rules - the $0 per-request savings are real and, at scale, substantial. It's a reason to budget for the ongoing maintenance the same way you'd budget for infrastructure, not treat "we wrote the rules once" as a finished task.

Where Rule-Based Routing Wins

Rules are the correct default whenever a category can be determined from something the application already knows with certainty: an authenticated account tier, a feature flag, an API endpoint, a UI element the user explicitly selected. In these cases, running a classifier isn't just wasteful - it's actively worse, since it introduces a probabilistic step to re-derive information the system already had deterministically.

Where It Doesn't

Open-ended natural language is the wrong target for a rule engine as the sole mechanism, for exactly the reason described above: phrasing varies in ways no fixed pattern list fully anticipates, and misses are silent. Products with rapidly evolving language - new slang, new feature names, new ways users describe the same underlying request - will see rule coverage decay continuously unless maintenance keeps pace, which is rarely sustainable past a handful of well-defined categories.

The AI PM's Qualification Checklist for Rule-Based Routing

As explored in the AI Product Management curriculum (Lesson 53), a well-designed guardrail architecture layers defenses across the input and output boundary rather than relying on any single mechanism - the same principle that governs where a rule engine belongs in a routing stack.

Before deploying a rule as the sole routing mechanism for a category, check it against four questions:

1. Is this a metadata rule or a content rule? Metadata rules (headers, account tiers, feature flags) are close to risk-free. Content rules (keyword and regex matches) need explicit review of what unrelated text could plausibly trigger them.
2. What happens on a false negative? If the answer is "it falls through to a smarter tier," the rule is safe to ship even if imperfect. If a miss means the request is silently mishandled with no fallback, the rule isn't ready.
3. What happens on a false positive? For hard-override rules specifically, a false positive routes a routine request to an expensive tier - wasteful, but not dangerous. For rules that route toward a less-scrutinized path, a false positive can be a genuine risk; audit those patterns harder.
4. Who owns keeping this rule table current, and how will they know when it's stale? A rule with no maintenance owner and no drift-detection process will decay invisibly. Treat the rule table as a living artifact with the same monitoring discipline applied to any other routing tier in the stack.

Rule-based routing is the cheapest, fastest, and most auditable technique in this entire series - and also the one most likely to quietly stop matching reality if nobody is watching it.