Choosing an AI Architecture

Match architecture to the job: rules, prompts, retrieval, tools, workflows, agents and fine-tuning.

01 / table

Start with the simplest sufficient pattern

Pattern	Good for	Main constraint
Deterministic rules	Known logic, permissions, calculations	Weak at ambiguity
Prompt-only	Drafting or classification with supplied facts	Context and freshness limit
RAG	Current or private evidence	Retrieval and permission quality
Tool calling	Live reads or bounded actions	Scope, validation, audit
Fixed workflow	Predictable multi-step tasks	Less flexible paths
Agent	Variable multistep investigation	Higher variability, latency, testing burden
Fine-tuning	Recurring behavior/style improvements	Not for fresh facts or permissions

02 / cards

Common false choices

RAG vs fine-tuning

Retrieve changing or private facts; tune when many task examples support improving behavior.

Workflow vs agent

Choose fixed sequences for known routes; agent only when variable planning earns the cost.

03 / flow

Evaluate each layer

Retrieval + accessModel qualityTool correctnessWorkflow outcomesLatency + cost

04 / note

Worked example

Policy Q&A: permission-filtered RAG. To change a customer record: add a narrowly scoped tool and explicit approval.

05 / list

Architecture questions

Is the gap missing facts, behavior, actions, or variable planning?
Can rules or fixed flows do it safely?
Where does authoritative state live?
What can the model actually change?

GOING DEEPER

Evaluate the layer that carries the risk

Evaluation map

Layer	What to verify
Retrieval	Relevant source found, freshness, permission filtering, ranking and provenance
Model output	Groundedness, completeness, uncertainty handling, structured fields and safety
Tools/actions	Correct tool, valid arguments, approval, scopes and side-effect recovery
End-to-end workflow	Task outcome, latency, cost, human review effort and fallback behavior

Specific design progression: For a policy Q&A assistant, start with retrieval against permission-filtered current policies. To update a customer account, add a narrowly scoped action tool, server-side validation and explicit authorization. Use an agent only if investigative paths genuinely vary enough to justify the additional autonomy.

Architecture boundary: Retrieval solves changing/private facts; fine-tuning can adapt repeated task behavior but not update authoritative knowledge. A fixed workflow is easier to audit than an agent when the sequence is known. Access controls and irreversible approvals must never rely on a prompt alone.

PRACTICAL FRAMEWORK / AI PRODUCT WORK

Missing facts, procedure or action?

What is missing?	Choose first	Watch for
Current, trusted facts	Permission-filtered retrieval (RAG)	Stale sources, access leaks, conflicting truth.
Repeatable method or checks	Skill, runbook, structured workflow or instructions	Skipped steps, weak exit criteria, bad recovery.
Live action or reading a system	Scoped tool/API with deterministic permission checks	Unsafe side effects, wrong arguments, lack of audit.
Variable multi-step route	Agent only when bounded planning earns its complexity	Runaway loops, weak oversight, cost and ambiguity.

Combine these when needed. A retrieval source does not grant authority; a procedure cannot replace missing evidence; a tool does not justify unattended action.