Chain of Thought: Return decision (fraud vs legitimate)
Idea: A customer requests a return. Instead of letting the model jump directly to "approve/reject", it must write an explicit reasoning chain like a support agent documenting a case for a supervisor.
Visual chain
[Return request: "Defect claim on headphones"]
|
[Phase 1: Facts only]
|
[Phase 2: Red flags]
|
[Phase 3: Legitimacy]
|
[Phase 4: Policy check]
|
[Phase 5: Final decision]
What Example 14 demonstrates
Facts: Extract data only, no early judgment.
Red Flags: Run explicit fraud screening checkpoint-by-checkpoint.
Legitimacy: Build the customer-side case as a balancing force.
Policy Check: Apply rules (window, value, history) before deciding.
Decision: Decide only after all prior phases are complete.
This structure makes the decision auditable and debuggable.
Why this matters in borderline cases
Without Chain of Thought, a borderline case often becomes a guess.
Example:
fraud score: 6/10
legitimacy score: 7/10
A direct one-shot classifier may flip randomly based on prompt phrasing.
With CoT, you can inspect each phase, identify the weak step, and fix the chain (for example policy interpretation or missing evidence handling).
The five CoT phases and their role
Phase
Why it exists
Facts
Prevent early bias by separating extraction from evaluation
Red Flags
Force explicit fraud risk checklist coverage
Legitimacy
Preserve customer fairness and avoid one-sided suspicion
Policy Check
Constrain model behavior with business rules
Decision
Produce a traceable outcome with clear rationale
Core takeaway
Chain of Thought does not just improve answer quality.
It improves governance:
supervisors can audit why a case was approved/rejected,
teams can spot where reasoning drift happened,
and policy changes can be reflected by updating one phase instead of rewriting the whole prompt.
CoT with reasoning vs non-reasoning LLMs
A common confusion: "If reasoning models like o3, DeepSeek-R1, or Qwen3 with thought mode enabled already reason internally, do I still need explicit Chain of Thought?"
There is no intermediate reasoning unless you build it.
CoT scaffolding creates the reasoning that would otherwise not exist.
Reasoning LLM (o3, DeepSeek-R1, Claude Extended Thinking, Qwen3 with thoughts: "auto"):
prompt -> hidden chain -> answer.
The model already produces internal reasoning tokens.
CoT scaffolding channels that reasoning into a fixed, inspectable shape.
Why explicit CoT is critical on a non-reasoning model
Without scaffolding, borderline cases (fraud 6/10 vs legitimacy 7/10) collapse into coin-flip behavior.
The model has no "place" to reason in, so it picks an answer-shape and back-fills justification.
Each of the 5 phases forces coverage that the model would otherwise skip - especially the legitimacy phase, which counters one-sided suspicion.
Schema grammar matters more here, because the model has fewer defenses against drifting outside the contract.
Why explicit CoT is still valuable on a reasoning model
Hidden internal reasoning is not auditable. Compliance, support QA, and incident reviews need a written trail, not an opaque "we trust the model".
Internal reasoning does not follow your taxonomy. Your fraud checklist, your policy rules, your refund workflow - all of these are domain-specific and absent from any pretraining corpus.
You cannot fix a step you cannot see. If a borderline case keeps going wrong, structured phases let you locate the weak link (for example: legitimacy reasoning is too soft) and improve only that prompt.
Internal reasoning varies between runs. Structured CoT produces a stable contract for downstream tooling (logging, analytics, escalation routing).
Public reasoning traces from reasoning models can be post-hoc rationalizations rather than the actual decision path. Treat them as a UX feature, not as audit evidence.
Practical recommendations
Reasoning model + light CoT: keep the 5 phases, shorten phase prompts, let the model reason inside each call. Lower verbosity, same auditability.
Non-reasoning model + heavy CoT: keep the 5 phases, expand phase prompts with checklists and examples, tighten schemas, lower temperature.
Hybrid model like Qwen3: pick a thought mode per phase. Use thoughts: "auto" on Phase 5 (Decision) where trade-offs matter, and thoughts: "discourage" on Phase 1 (Facts) where extraction is mechanical.
Anti-patterns
Telling a reasoning model to "think step by step" inside the prompt - redundant token spend, and it can derail the model's own internal chain.
Using a non-reasoning model for borderline decisions without CoT - results are not reproducible, not defensible, and not safe in production.
Trusting raw reasoning traces from reasoning models as audit evidence - they look convincing but are not policy-compliant by construction.
Comparing confidence values across model classes - calibration differs sharply; treat confidence as model-internal only.
Bottom line
CoT is not a substitute for a reasoning model, and a reasoning model is not a substitute for CoT. They solve different problems:
Reasoning models improve raw answer quality.
Chain of Thought turns whatever reasoning happens into a governable workflow.
For high-impact decisions, you usually want both.
When to use CoT in real work
Use Chain of Thought when decisions are high-impact and need reviewability.
System admin mental model
A deployment request looks risky, but not obviously wrong.
Facts: current load, recent incidents, rollback readiness.