Module 3 • Lesson 10• 50 mins

Fine-tuning - Retraining Model Reflexes with Custom Data

Model parameter adaptation in Fine-tuning, viable vs anti-pattern use cases, and the 4-tier AI customization escalation ladder for PMs.

Distinguishing In-context (Prompt/Skill/RAG) vs. Parametric Adaptation (Fine-tuning)

Evaluating viable vs. anti-pattern fine-tuning use cases across cost, data, and latency

Mastering the 4-tier AI customization escalation discipline

Fine-tuning - Retraining Model Reflexes with Custom Data

RAG and Skills share a common principle: they do not alter the base model itself, modifying only what enters the context window at runtime. Fine-tuning takes a fundamentally different path: directly modifying internal parameters to turn domain knowledge and style into intrinsic model reflexes.

1. What Is Fine-tuning: Modifying Internal Model Parameters

Mechanics Comparison:

In-context (Prompt / RAG / Skill): An employee brings SOP playbooks and reference books into an exam room - working from external documents (Base model remains 100% frozen).
Fine-tuning (Parametric Adaptation): An employee undergoes intensive apprenticeship across tens of thousands of cases - internal thinking reflexes are reshaped without needing manuals (Internal model weights are modified).

In-Context Learning vs Model Fine-tuning

RAG and Skills alter what the model 'sees' in the Context Window. Fine-tuning alters the internal 'brain' weights of the model.

Toggle between the two approaches to compare their operational mechanics:

Architecture Flow

[External Docs / SOP] ──► [Context Window] ──► [Frozen Base Model] ──► [Output]

Base model parameters remain untouched. All proprietary knowledge is streamed dynamically through the prompt.

Can thiệp vào đâu?

Modifies only the prompt payload passed into the Context Window per API call (Base model is 100% frozen).

Thời gian triển khai & Dữ liệu

Instantaneous (Minutes to hours of prompt authoring / doc indexing).

Cơ cấu chi phí

Per-call token costs (paying repeatedly for long system prompts and retrieved context).

Khả năng thay đổi (Agility)

Extremely agile: Policy updates take effect immediately.

Ẩn dụ trực quan: An employee referencing an SOP handbook and library books during an open-book exam. The employee remains unchanged.

In-context approaches excel for dynamic knowledge; Fine-tuning excels for proprietary voice and high-volume token efficiency.

Workflow: Prepare thousands of curated prompt-completion pairs (Standard Input → Ideal Target Output) and run additional gradient training on GPU infrastructure.

2. When to Choose and NOT Choose Fine-tuning

Common Misconception: "To make AI understand company knowledge, we must fine-tune it." This is an expensive trap in time and compute.

DO NOT Fine-tune when:
- Data changes frequently: Price sheets, promotions, stock levels (requires continuous dataset prep and retraining).
- Source citations are required: Models absorb knowledge into latent intuition, unable to cite exact clauses or page numbers (RAG is far superior).
DO Fine-tune when:
- Embedding distinct brand persona: Standardizing company tone across 50,000 top-tier support dialogs without long prompts.
- Enforcing strict output structure: Ensuring AI always returns the exact required table or file format, without adding or removing fields.
- Slashing token costs & latency: Replacing a 2,000-token System Prompt with concise instructions across millions of high-throughput calls.

3. The 4-Tier Customization Matrix for Product Managers

Approach	Intervention Layer	Core Strength	Primary Trade-off	When to Choose?
1. Buy / Prompting (call commercial APIs directly)	Context window	Fastest, $0 engineering overhead	Prone to drift, high token cost	Prototyping, simple single-turn tasks
2. Skills	Workflow & Constraints	Standardizes flow, bounds risk	Requires domain expert maintenance	Complex multi-step business procedures
3. RAG	External knowledge store	Real-time updates, citations	Dependent on search retrieval quality	Large datasets, volatile facts
4. Fine-tuning	Internal model weights	Unique tone, lower latency	High compute cost, rigid updates	Distinct persona, massive-scale classification

The 4-Tier AI Customization Escalation Ladder

Golden rule: Always start at the simplest, lowest-cost tier (Tier 1) and escalate only when facing genuine capability limits.

PM Escalation Discipline

Start with Prompting → Missing procedures? Package a Skill → Missing facts? Implement RAG → Need custom brand reflex or scale? Fine-tune.

Click each tier to inspect trade-offs and escalation triggers:

1. Prompt Engineering

Lowest Cost · Fastest

Ưu tiên sử dụng cho:

Prototyping, single-turn tasks, generic knowledge synthesis.

Điểm mạnh lớn nhất:

Shipped in minutes, zero backend infrastructure, instant iteration.

Đánh đổi / Hạn chế:

Bound by context window limits; prone to drift across long multi-turn sessions.

Khi nào nên leo thang cấp độ tiếp theo? When the model knows the facts but misses mandatory steps or formats → Escalate to Tier 2 (Skills).

Over 80% of enterprise AI features are solved elegantly within Tiers 1–3 without needing expensive model fine-tuning.

4. PM Case Study: Diagnosing Prompt Stuffing vs. Fine-tuning

Flawed Claim: "Our system is fine-tuned: every query pastes 200 pages of policy agreements into the prompt header."

Diagnosis: This is raw Prompt Stuffing (In-context), causing context overflows, severe latency, and extreme per-call token costs.
Correct Architecture: Use RAG to pull 1–2 relevant policy clauses + use a Skill to format clean, accessible answers.

Hands-on Workshop70 minutes

Designing an AI Customization Strategy for EcoCart

RAG (Lesson 8): Design an International Returns & Warranty Assistant (500 pages) specifying Retrieval, Context assembly, and citation Generation.
Skills (Lesson 9): Package a Skill for Lost Shipment Resolution: Define Trigger, 3-step SOP, and 2 Negative Constraints.
Fine-tuning (Lesson 10): Evaluate classifying 500,000 daily reviews into 25 sentiment categories: Analyze 2 benefits regarding cost and latency.
Architecture Matrix: Map all 4 customization approaches (Buy / Prompting, Skill, RAG, Fine-tune) across 4 EcoCart product features with rationale.