Agentic architecture, evaluation, reliability, and security

Agentic systems combine planning, tools, and iteration. Common patterns: routing (send each input to the right handler), orchestrator‑workers (a lead delegates bounded sub‑tasks), and evaluator‑optimizer (generate → critique → improve). Evaluate with a mix of ground‑truth checks and LLM‑as‑judge scoring against explicit rubrics — you can't improve what you don't measure. Build for reliability: retries with exponential backoff, idempotency so retries are safe, and graceful degradation. Add observability (log inputs, tool calls, outputs) and guardrails (input/output validation, prompt‑injection defence, human‑in‑the‑loop for high‑stakes actions). Security first: treat every tool input as untrusted and scope every capability.

Injection is an input‑trust problem: retrieved documents, web pages, and tool results may contain adversarial instructions. Defences stack: delimit untrusted content (tags) and tell Claude explicitly that it's data, not instructions; never let retrieved content override the system prompt; allowlist tools per task so a hijacked step can't reach dangerous capabilities; validate outputs against schema and policy; and gate irreversible actions behind human approval. Least privilege means even a successful injection has a small blast radius.

• Patterns: routing · orchestrator‑workers · evaluator‑optimizer · prompt chaining.
• Evals: define a golden set; combine ground‑truth checks with LLM‑as‑judge + rubric.
• Measure before tuning: baseline → change one thing → compare.
• Reliability: retry with exponential backoff + jitter · make actions idempotent · degrade gracefully.
• Observability: log prompts, tool calls, results; trace multi‑step runs.
• Guardrails: validate inputs/outputs · defend against prompt injection · human approval for high‑stakes.
• Security: least privilege · treat tool inputs as untrusted · never expose secrets to the model.
• Human‑in‑the‑loop: gate irreversible actions.

• Start from failure modes — what going wrong looks like defines what to measure.
• Build a golden set including edge cases; keep a held‑out portion you never tune on.
• Ground truth where objective; LLM‑as‑judge + explicit rubric where subjective.
• Validate the judge against human labels before trusting it.
• Report per‑domain, not just aggregate — averages hide regressions.
• Baseline → change one variable → compare. Run evals in CI to catch drift.

“REPS” to ship an agent you can trust — Reliability (retries/idempotency), Evaluation (rubrics + ground truth), Protection (guardrails + injection defence), Security (least privilege). Skip a letter and production will find it for you.

Keep production agents on the “RAIL” — Retries with backoff (+ jitter), Approvals for high‑stakes actions, Idempotent operations, Logs and traces. If it's not on the RAIL, it derails in production.