AgentsEvalsGoogle AIAI Agent Skill
google-agents-cli-eval
Agent task evaluation framework calculating Pass@k, trajectory correctness, and tool invocation fidelity.
Install Options
Install into Claude Code, Cursor, Codex, or Antigravity with:
$ npx skills add google/agents-cli@google-agents-cli-eval
Real Work Situation Solved
Build an AI eval benchmark from scratch with zero sample data
“Engineers ask "how do we measure model accuracy?" before launch - where do you get your first test cases?”
5-Step Action Framework
- 1Handcraft 20 core user scenarios (10 standard asks, 5 ambiguous/adversarial probes, 5 out-of-scope queries).
- 2Define the ground truth target answer and a pass/fail rubric for each test case.
- 3Have 2 teammates evaluate the test independently - if they disagree, your rubric isn't objective enough yet.
- 4Run the model through all 20 cases and record the baseline pass rate before tuning prompts or RAG.
- 5Append a new test case every time you find an unexpected failure or weird output during dogfooding.
Before
"Wait until launch and collect real user queries to evaluate."
After
"Start with 20 curated golden test cases and clear rubrics (10 happy + 5 edge + 5 adversarial), measure 65% baseline pass rate, target 85% for beta release."
Related Course Lesson
AI Agents for PM • Lesson 11
SKILL.md Source Instructions
SKILL.md • Read-only preview# Google Agents CLI Eval Run evaluation loops over agent trajectories, trace tool calls, and compute statistical confidence intervals.