Action-Centered Threat Modeling & Execution Isolation

Trust must be designed into the workflow

By Day 3, the capstone agent can use tools and load domain procedures. Those capabilities also increase its blast radius. A model becomes an agent only when a harness gives it state, tools, execution, and feedback. Security must therefore protect the whole harness, not just the prompt or the model endpoint.

Traditional software often treats trust as a gate: authenticate once, receive a role, then execute predictable code. Agent behavior is probabilistic and context-dependent. A valid credential does not prove that the current action matches the user's intent. Trust must be checked continuously against identity, context, tool, data, time, and proposed impact.

The secure agent framework below organizes that responsibility across seven pillars.

Read the framework from foundation to runtime response:

1. Infrastructure and networking contain generated code and govern outbound access.
2. Data protects sensitive context, memory, and tenant boundaries.
3. Model protects system instructions and filters semantic attacks.
4. Application and runtime intercept tool calls and enforce deterministic hooks.
5. Identity and access assign explicit agent identities and short-lived permissions.
6. Observability and security operations reconstruct behavior and respond to anomalies.
7. Governance assigns accountability, evidence, and release obligations.

A Product Manager does not implement each control, but does decide which user actions need them. Security requirements become useful only when mapped to the journey. "Use least privilege" is too abstract. "The research agent can read synthetic account data for the current workspace and cannot write to CRM" is testable.

The seven pillars are a defence-in-depth system, not seven independent checklist items. A failure in one pillar must still meet a control in another:

This architecture moves the perimeter from static identity to context as the perimeter. Its runtime output is Effective Trust: not a permanent badge, but a decision recalculated from supply-chain integrity, identity, current associations, behavior, and requested impact. The PM must define what evidence can raise or lower trust and which missing signal causes a fail-closed decision.

Build a threat model around actions

Start with the actions the product can take, not a generic list of AI threats. For every tool or delegated task, document:

1. Authority: whose permission is being used.
2. Scope: the smallest data and action boundary required.
3. Impact: what changes if the action succeeds.
4. Reversibility: whether the result can be undone.
5. Visibility: what the user and operator can inspect.
6. Intervention: where a person or policy can stop the workflow.

Then trace how untrusted context could influence that action. A malicious instruction can enter through a retrieved webpage, repository file, MCP response, support ticket, or user-uploaded document. The model may not distinguish that content from legitimate guidance unless the harness separates data from instructions and validates the resulting action externally.

For the sales research capstone, a useful risk ladder might be:

This table keeps autonomy tied to evidence. The product can deliver value through research and drafting without granting broad write access on day one.

Turn the threat model into a PM-owned action register, not a slide that is reviewed once and forgotten. Each row should name the user job, tool, data classification, maximum impact, approval owner, rollback method, and evidence retained. Update it whenever a tool, model, retrieval source, or permission changes. Useful operating metrics include blocked high-impact actions, actions executed with stale approval, mean time to revoke access, percentage of tool calls covered by policy, and rollback success rate.

Contain execution and the supply chain

Generated code should run in an ephemeral, low-privilege environment with restricted network access. A sandbox is not a complete security strategy, but it limits how far a bad script can reach and prevents state from quietly persisting across runs.

Dependency installation deserves its own control. Models can propose outdated, incorrect, or nonexistent packages. Attackers can publish malicious packages under plausible names. Use approved registries, pinned versions, software composition analysis, and deterministic CI checks before any generated artifact advances.

Network egress should travel through governed paths. A broad domain allowlist does not prevent an agent from retrieving malicious instructions from an allowed site. Prefer sanitized retrieval services, internal proxies, or offline caches for untrusted external content. Keep credentials out of prompts, logs, generated scripts, and test fixtures.

This is non-interactive internet access: the agent requests content through a governed retrieval path but cannot freely browse, authenticate to arbitrary sites, or follow instruction-bearing links. The product must define allowed content classes, cache freshness, sanitization failures, and a user-visible fallback when a source cannot be fetched safely.

For application changes, separate advisory feedback from enforcement. IDE guidance can help a developer notice risky code, while CI remains the hard gate for vulnerable dependencies, secret leaks, missing tests, and structural security rules. An agent should not be able to talk a deterministic pipeline out of failing.

Make the containment contract specific. The sandbox must block raw host access, isolate the network, use only task-scoped credentials, and reset all writable state between runs. A container that persists agent-created files or shares the parent credential is not ephemeral. Test container escape, forbidden file reads, forbidden egress, credential expiry, and clean reset; record the result against the sandbox version.

The supply-chain failure has a specific shape: the model invents a plausible package, an attacker publishes malware under that name, and the agent installs it automatically. This is slopsquatting. Prevent it with approved registries, cryptographic version pinning, SBOM verification, package signatures, SCA, and binary authorization before promotion. The PM sets the dependency policy and exception owner; evidence includes unknown-package blocks, unsigned artifacts, vulnerable-package escape rate, and time to revoke a compromised version.

Application security also needs explicit negative cases. Generated UI code often places API keys, password checks, or session flags in the browser, where a user can inspect or modify them. Generated backends often omit default-deny authorization or row-level security, exposing private records or staging systems. Add tests that manipulate client state, call the backend without the UI, cross tenant boundaries, and access staging anonymously. IDE advisory linting can explain the issue quickly, but deterministic SAST, SCA, secret scanning, authorization tests, and deployment policy remain the CI/CD gate.

Repositories are also input channels, not trusted ground truth. An attacker can hide an invisible payload in code, documentation, or retrieved examples using zero-width Unicode, homoglyphs, misleading comments, or instruction text designed for an agent rather than a human reviewer. Once copied by an agent, repository poisoning can spread across many files faster than line-by-line review can detect it.

Treat repository ingestion as a security boundary:

• Normalize and scan Unicode before content enters agent context.
• Distinguish executable instructions from quoted data and documentation.
• Record the source and integrity of retrieved snippets and dependencies.
• Block generated changes that introduce unapproved packages, encoded payloads, or unexplained network calls.
• Re-scan the complete diff in CI, even when the IDE reported no issue.

The PM decision is not which scanner to buy. It is which sources the product promises to trust, what happens when provenance is missing, and whether a suspicious source blocks the run or merely removes that source from context.