Troubleshooting & Optimization

Troubleshooting and Optimization is the lightest domain on this exam at 10 percent — roughly six of the sixty items — and the only one that is purely about recovery. Almost nothing here is new material: each item hands you a workflow that has already gone wrong and asks what you change first.

The idea that unlocks it is that a weak answer is a symptom, and it almost always has one of four causes with a cheapest-first fix order. The request was unclear. The source material was never supplied. The task was too big for one turn. Or the job is genuinely outside what the tool can do. Diagnose before you act — candidates who reach first for a bigger model tier or one more retry are buying an expensive fix for a cheap problem.

The second idea is knowing when to stop. Two or three deliberate attempts, each changing a single variable, tell you whether the prompt is the problem. After that, more prompting is rework, and the real answer is a different approach, a human reviewer, or a system that holds the data. Given the weighting, bank the sequence and study elsewhere.

• Diagnose before you retry. A weak answer has one of four causes: unclear request, missing source material, task too big for one turn, or genuinely beyond the tool. Name the cause before you touch anything.
• Fix cheapest-first: sharpen the ask, attach the source, split the task, change the approach, escalate to a human. A more capable model tier is rarely the first move.
• Change one variable per attempt. Rewriting the prompt, attaching the file and splitting the task all at once tells you nothing about what actually worked.
• "Make it better" is not a correction. Name the specific defect, point at the offending line, and say what good looks like.
• Same prompt, same weak output? Change kind, not degree — add a worked example of a good answer, supply the missing input, or reframe the audience and purpose. Repeating yourself more forcefully changes nothing.
• An ignored instruction was usually buried mid-conversation, contradicted by something earlier, or one of too many simultaneous constraints. Restate it at the point of the request and move standing rules somewhere persistent.
• Same request, different answers is normal, not a defect. Constrain it: give a template or example of the exact output shape, state the format explicitly, supply identical source material, and save the wording for reuse.
• Restart a conversation when answers regress, discarded versions reappear, or corrections stop sticking. Carry forward the agreed requirements and the source — not the argument.
• Front-load the brief: audience, purpose, format, length, tone, must-include, must-avoid, source material, and what "good" means. One clear brief costs less than three rounds of correction.
• For a large deliverable, ask for an outline or a single section first. Approve the shape before paying for the whole draft.
• Match tuning effort to reuse. A one-off email deserves none. A weekly report that ten people run deserves a saved, tested prompt.
• When someone says it does not work for them, ask for the exact prompt and the exact output, then reproduce it yourself before concluding anything.

The tempting wrong answers on this domain, and the mistakes people actually make.

• Escalating the model tier first. It is the most attractive distractor in the whole domain. Reach for more capability when the task is genuinely hard reasoning — not when the brief was thin or the file was never attached.
• Calling it a hallucination when you never supplied the document. If the source was not in front of it, that is a missing-input failure. The fix is the attachment, not a lecture about accuracy.
• Hitting retry on an identical prompt. Natural variation may hand you a better draft once. That is luck, and it will not survive the next run.
• Piling correction onto correction in a thread that has already gone wrong. Past the third failed round the accumulated context is working against you.
• Restarting, then pasting the whole old conversation back in. You just re-imported the confusion you were escaping.
• Sunk cost. Time already spent in a conversation is never a reason to keep it alive.
• Treating run-to-run variation as a bug to report rather than a property to design around with templates, fixed formats and a review step.
• Optimizing a prompt that is doing exactly what you asked. Reread your own request before you decide the answer is wrong.
• Raising a ticket before reproducing it once yourself with the same request and the same inputs.
• Assuming everything is a prompting problem. Live system access, a figure only the system of record holds, and a judgement someone has to sign are not fixable by wording.
• Fixing it privately. If the request was the problem and the task recurs, save the corrected version where the team can find it or the same complaint comes straight back.
• Over-studying this domain. At 10 percent its items are recovery scenarios borrowed from the other six. Learn the diagnostic sequence, then go spend the time on output evaluation.

RESET — what to do when the output is wrong. The word is also the last resort: when a conversation has drifted beyond repair, the fix is literally a reset.

• R — Reproduce. Run it again unchanged. A failure that repeats is a design problem. A failure that comes and goes is a specification problem — you have not pinned the output down.
• E — Examine the cause. Unclear request, missing source, task too big, or beyond the tool. Pick one before you change anything.
• S — Split. Cut the deliverable into one output per turn. Much of what looks like poor quality is simply overload.
• E — Edit one thing. One variable per attempt, or you will never know which change worked or how to repeat it.
• T — Test and time-box. Check the result against the acceptance criteria you wrote up front. Two or three attempts with no movement means stop prompting and change the approach.