ProductDecode
Module 4 • Lesson 12• 50 mins

How Does an Agent "Decide" What to Do Next?

The ReAct reasoning loop (Thought → Action → Observation), conceptual tool calling, the 4 levels of autonomy, and safety circuit breakers.

The ReAct reasoning loop: Thought → Action → Observation
The 4 levels of autonomy and Human-in-the-Loop guardrails
Safety guardrails: Max steps budget, Circuit Breakers, and Token caps

How Does an Agent "Decide" What to Do Next?

How does an AI model know which tool to call first, which to call next, and when a task is actually finished? The answer lies in the ReAct (Reasoning + Acting) Execution Loop - the core cognitive pattern enabling Agents to reason, act, and dynamically adapt based on real-time feedback.

1. The ReAct Reasoning Loop: Thought → Action → Observation

Unlike rigid A → B → C scripts, an Agent reasons adaptively across a 3-step loop:

  1. Thought: The AI evaluates goals and environmental state: "To achieve this goal, what is my next action?".
  2. Action: Selects a tool from its catalog and dispatches an execution command (e.g., query warehouse API).
  3. Observation: Evaluates returned ground truth: "What did the tool return? Does this advance the goal?".

The ReAct Reasoning Loop (Thought → Action → Observation)

An Agent does not follow rigid scripts. It reasons, invokes tools, and inspects observations to dynamically adapt.

Initial Goal Assigned to Agent:

“Customer reports order #7124 is 45 minutes late. Investigate status, resolve the bottleneck, and compensate if fault lies with the platform.”

Click through each Turn to trace the Agent's cognitive flow:
01. Thought (Reasoning):

I need to inspect the courier's real-time GPS coordinates and determine why order #7124 is stalled.

02. Action (Tool Invocation):
get_driver_location(order_id='7124')
Parameters: {"order_id": "7124"}
03. Observation (System Feedback):

Return data: Courier experienced a flat tire at Nguyen Thi Minh Khai St; stationary for 35 minutes.

After each Observation, the Agent enters a new Thought cycle until the overarching goal is fully achieved.

Execution rule: After each Observation, the Agent enters a new Thought cycle until the overarching goal is resolved, then outputs a final report.

2. Conceptual Tool Calling: How Does AI Command Software?

The agentic orchestration process follows 3 steps:

  1. Tool Definition: Engineers define available Tools with clear schemas (e.g., cancel_order requiring order_id and reason).
  2. Structured Invocation: Instead of prose, the model outputs structured JSON: {"action": "cancel_order", "parameters": {"order_id": "9821"}}.
  3. Execution & Feedback: Backend processes the JSON command, mutates database state, and passes results back to the AI for next-step reasoning.

3. The 4 Levels of Autonomy in Products

LevelTier NameOperational MechanicEcoCart Example
Level 1Advisory (Suggestion Only)AI analyzes & recommends; humans execute manuallySuggests 3 top sellers for homepage banner
Level 2Human Approval (HITL)AI stages actions; pauses for human sign-offStages $50 refund waiting for CS approval
Level 3Supervised AutonomyAI runs low-risk actions; dispatches summary logsAuto-cancels unpaid carts after 24h & sends email
Level 4Full AutonomousAI runs silently in background without turn alertsSyncs inventory counts across 5 hubs every 10 mins

The 4 Levels of AI Agent Autonomy in Products

Autonomy boundaries define the operational freedom granted to an Agent based on error tolerance and business stakes.

Golden Rule for PMs:

Any action mutating financial balances, legal terms, or permanently deleting data must be locked at Level 2 (Human Approval).

Click each level to inspect operational mechanics and real-world implementations:

Level 2: Human Approval (Human-in-the-Loop)

Moderate (Gated before real-world mutation)
Human Role:

Reviews the staged payload and clicks 'Approve' or 'Reject'.

Agent Authority:

Prepares 100% of the action payload and pauses execution for sign-off.

EcoCart Production Example:

Agent stages a $75 refund for a damaged TV screen, pausing for CS manager sign-off.

Hướng dẫn triển khai cho PM: Mandatory for all financial refunds, contract modifications, or sensitive customer state mutations.
Higher autonomy tiers create seamless user experiences but demand rigorous evaluation datasets and automated guardrails.

Golden Rule for PMs: Any action mutating financial balances, legal terms, or customer data must be locked at Level 2 (Human-in-the-Loop).

4. Mitigating Infinite Execution Loops

Failure mode: API network timeout → Retry → Timeout → Loops hundreds of times. Consequence: Depletes token budgets and freezes backend services.

Hands-on Workshop70 minutes

Designing an Autonomous Order Resolution Agent for EcoCart

  1. Tool Arsenal Design (Lesson 11): Specify 4 essential Tools (GPS lookup, courier messaging, voucher generator, order status updater).
  2. ReAct Cycle (Lesson 12): Write out a 3-turn ReAct sequence (Thought → Action → Observation) from delay complaint to resolution.
  3. Autonomy Tier Allocation (Lesson 12): Map each Agent action to its appropriate level (Level 1 through Level 4).
  4. Safety Guardrails: Define the Max Step budget and failover protocols when courier GPS coordinates disconnect.