Operational Boundaries: Structural vs Semantic Gating & Context Hygiene
Separating structural policy from semantic policy, enforcing context hygiene and prompt sanitization, combining deterministic tests with behavioral evals.
Build the operational boundary
A production decision needs more than a successful demo. The operating model must specify:
- Allowed, draft-only, approval-required, and blocked actions.
- Context hygiene for retrieved and user-provided content.
- Identity and permission scope for every tool.
- Sandboxing and network boundaries for generated code.
- Human review for high-impact actions.
- Logs that connect user intent to tool activity and output.
- Deterministic tests plus behavioral evaluation.
- Owners for incidents, evaluations, skills, tools, and policy.
- A staged rollout gate and a tested rollback path.
Use the source incident as a pre-mortem: an agent asked to create a button autonomously clicked it, found a deprecated email endpoint from context, and sent hallucinated messages to 50 colleagues. Each local step looked plausible; the chain failed because target information was missing, legacy context was available, browser actions were auto-approved, and no policy or HITL gate stopped external communication. The PM response is to remove deprecated endpoints from context, require an explicit target, keep UI testing in an isolated non-production profile, block email structurally in development, and require semantic policy plus a concrete approval receipt elsewhere.
Guardrails should sit outside the model when possible. Prompts can guide behavior, but they are not tamper-proof enforcement. A policy layer can combine deterministic and semantic checks.
Separate structural policy from semantic policy
Structural checks answer binary questions quickly: Is this tool allowed for this role? Is email blocked in the local environment? Is a write action permitted during this rollout stage?
Semantic checks inspect how an allowed capability is being used: Does this email contain sensitive data? Does the proposed CRM update match the approved user intent? Is retrieved content trying to override system policy?
Follow one proposed action through the policy path. The sales agent produces propose_crm_update with sanitized fields, source IDs, and user intent. The policy server first checks the structural facts: this role may draft in the pilot, the environment is non-production, and direct commit is blocked. It then checks semantics: the fields are supported by evidence, contain no unmasked PII, and match the approved account. Only then does the tool run and return a receipt. A later send_email call fails immediately at the structural layer; rephrasing the request cannot bypass the same tool prohibition.
Structural policy is fast, deterministic, and easy to audit, but cannot understand every harmful use of an allowed tool. Semantic policy understands intent and content, but is probabilistic, slower, and can produce false positives. Keep high-confidence architectural prohibitions structural; use semantic checks for content-dependent risk; route uncertainty to HITL rather than silently allowing it. Measure decisions and errors separately for each layer.
If either check fails, return a useful policy result to the agent and user. State what was blocked, which constraint applied, and what safe alternative is available. Do not allow the agent to bypass the decision by rephrasing the same action.
Make the zero-trust safety net explicit as independent layers:
| Layer | What it prevents | PM artifact or metric |
|---|---|---|
| Guardrail and policy server | A permitted model choosing a forbidden tool or unsafe argument | Policy matrix, block reasons, bypass and false-positive rate |
| Sandbox and egress boundary | Generated code reaching host files, production systems, or arbitrary networks | Sandbox profile, denied escape attempts, environment reset evidence |
| HITL checkpoint | High-impact action executing without informed consent | Risk-tier map, approval receipt coverage, approval fatigue |
| Test coverage | Deterministic regressions and repeat failures | Failing reproduction first, changed-code coverage, mutation or contract results |
| Behavioral evaluation | Intent, trajectory, quality, and recovery drift | Versioned rubric, baseline delta, critical-case pass rate |
| Context hygiene | PII, legacy endpoints, or injected instructions entering an action | Redaction rate, unresolved placeholder blocks, source provenance |
| Observability and rollback | Silent failure or an incident with no reconstruction path | Trace completeness, time to contain, rollback exercise result |
The policy server should intercept every consequential tool call before execution. Structural gating answers whether this role may use this tool in this environment. Semantic gating inspects whether otherwise allowed arguments violate privacy, intent, or content policy. It should return a stable reason code, user-safe explanation, retry guidance, and audit event. Version the policy alongside the eval cases so a policy change cannot silently expand autonomy.
HITL gates belong at risk transitions such as production deployment, schema change, financial transaction, external communication, or irreversible deletion. Present the sanitized intent, exact target, diff, evidence, and rollback option. Approval must authorize one concrete action, not the rest of the session.
For AI-generated test coverage, require the reproduction or failing test before the fix and prevent the same agent batch from weakening the assertion. Tests answer binary contract questions; evaluation uses rubric scores and to catch behavioral drift. Neither substitutes for the other.
Implement each layer independently. Run generated commands in an ephemeral low-privilege sandbox, isolated from host files and the primary network, with limited credentials and disposable state. Put HITL checkpoints before production deployment, schema change, financial transaction, and external communication. A reviewer sees sanitized intent and exact effect, not raw hidden context. Ask the agent to generate broader test coverage, but retain the failing reproduction and protect tests from being weakened in the implementation batch.
Evaluation covers what deterministic tests cannot: an agent may pass every tool unit test and still choose the wrong tool, paraphrase away a critical fact, or hallucinate evidence. Use versioned 0–5 rubrics, calibrated LLM-as-judge or agent-as-judge scores, trajectory checks that tolerate harmless ordering variance, and a release margin against the baseline. Record judge version, calibration agreement, score distribution, critical-case floor, and quality regressions hidden by an average.