BenchmarkingCompetitive AnalysisMetricsAI Agent Skill
ab-testing
Comparative feature testing, experimental teardowns, and defensibility benchmarks.
Install Options
Install into Claude Code, Cursor, Codex, or Antigravity with:
$ npx skills add coreyhaines31/marketingskills@ab-testing
Real Work Situation Solved
Deconstruct a competitor's AI feature without falling for marketing hype
“A competitor announced a "100% Autonomous AI Agent" - how do you reverse-engineer what's real under the hood?”
5-Step Action Framework
- 1Test adversarial edge cases, malformed inputs, and out-of-domain requests to find where the system breaks.
- 2Observe latency and rendering patterns: streaming implies an LLM call; instant response implies caching or rule-based logic.
- 3Determine whether the feature relies on proprietary model training or is simply a thin prompt wrapper around a commodity LLM API.
- 4Identify human-in-the-loop boundaries: where the workflow pauses for manual user confirmation.
- 5Compile a 4-factor matrix: core user problem solved, observed real-world accuracy, estimated unit cost, and actual defensibility.
Before
"The competitor's AI looks revolutionary, we must copy it immediately."
After
"Live teardown: feature is a simple prompt wrapper on GPT-4o with 6s latency that fails 4/10 numeric tasks. Proposal: solve with local rule engine in 50ms instead of blind imitation."
Related Course Lesson
AI Product Management • Lesson 8
SKILL.md Source Instructions
SKILL.md • Read-only preview# A/B Testing & Teardown Skill Run head-to-head capability evaluations comparing baseline implementations against competitors.