Tech EvaluationDecision MakingArchitectureAI Agent Skill
evaluating-new-technology
A framework for deciding whether to adopt a new AI/model-based approach or stick with an existing deterministic solution, based on real cost and risk.
Install Options
Install into Claude Code, Cursor, Codex, or Antigravity with:
$ npx skills add refoundai/lenny-skills@evaluating-new-technology
Real Work Situation Solved
Deterministic rules or an AI model - how do you decide?
“The problem in front of you could be solved with if-else or with AI - which do you pick, and why?”
5-Step Action Framework
- 1Ask: can the rule be written as a finite list that doesn't change with context? If yes → use a deterministic rule.
- 2Ask: does the system need to explain its decision to users or auditors? Rules are easier to explain than models.
- 3Ask: is the input varied, ambiguous, or does it keep shifting over time? If yes → a model is the better fit.
- 4Compare the cost of being wrong: a bad rule is a one-line fix; a bad model needs retraining and re-evaluation - which risk can this problem tolerate?
- 5When unsure, start with a rule for the 80% common cases and reserve AI for the harder-to-predict remainder.
Before
"Just use AI, it'll probably be more accurate."
After
"Approving refunds under $20: a deterministic rule (clear conditions, must be explainable to customers). Fraud detection: an AI model (input is varied, patterns keep shifting)."
Related Course Lesson
AI Literacy • Lesson 1
SKILL.md Source Instructions
SKILL.md • Read-only preview# Evaluating New Technology Skill Weigh a new technology's maturity, switching cost, and failure risk against the deterministic status quo before recommending adoption.