先看這個問題為什麼值得解Why this problem is worth solving
情境先行Start with the situation
想像你讓兩個 agent 訂一趟研究旅行。A 寫了兩頁計畫，B 先查日期、驗證價格，再保留可取消方案。哪一個比較會推理？ Imagine two agents planning a research trip. A writes two pages of plans; B checks dates and prices, then keeps a cancellable option. Which one reasons better?
如果只量文字長度或最後成功率，規劃、工具可靠度與環境運氣會被混在一起。 If we measure only trace length or final success, planning, tool reliability, and environmental luck become confounded.
先備知識Prerequisite
- 知道語言模型以 token 逐步生成 Know that language models generate token by token
讀完你能做到After this path
- 畫出代理系統的輸入、狀態、動作與回饋迴圈 Draw an agent loop with input, state, action, and feedback
- 區分推理長度與推理品質 Distinguish reasoning length from reasoning quality
- 找到評測中可能被環境或工具洩漏的訊號 Spot evaluation signals leaked by environments or tools
閱讀框架Concept frame
- 核心問題Question
- 一個模型完成較長的思考，是否就代表它更會解題？ Does a longer reasoning trace necessarily mean better problem solving?
- 對照軸Comparison axis
- 外部回饋的可靠度 External feedback reliability
- 判讀方式Evidence to inspect
- 並排閱讀三種相對情境，再回到論文檢查消融實驗、反事實測試與適用範圍。Read three relative scenarios side by side, then inspect the paper's ablations, counterfactual tests, and scope statements.
回答後查看解析Answer, then view the explanation
你的預測Make a prediction
在工具偶爾回傳錯誤結果時，哪個改動最可能穩定提升成功率？ When tools occasionally return wrong results, which change is most likely to improve reliability?
看現象，不急著講故事Observe before narrating
三個診斷訊號Three diagnostic signals
這些是待檢查的診斷假說。下一步用條件比較、消融實驗或反事實測試排除其他解釋。These are diagnostic hypotheses to test. Use condition comparisons, ablations, or counterfactual tests to rule out alternatives.
一次看完三種情境Read three scenarios side by side
三種情境對照Three-scenario comparison
當 feedback 可靠度改變，你會把 token 預算放在哪一層？ As feedback reliability changes, where should the token budget go?
對照軸：Comparison axis: 外部回饋的可靠度 External feedback reliability
需要交叉核對、回復點與保守停止條件。 Cross-checks, checkpoints, and conservative stopping become essential.
規劃與驗證都會影響結果；應做分層 ablation。 Both planning and verification matter; use layered ablations.
可更專注於規劃品質，但仍要防止 benchmark 洩漏。 Planning quality becomes central, while benchmark leakage still matters.
從直覺到機制From intuition to mechanism
把機制講清楚Explain the mechanism
Agent 不是一段很長的回答，而是一個會讀取世界、改變世界、再檢查世界的迴圈。 An agent is not one long answer; it is a loop that reads, changes, and re-checks the world.
輸入是任務與環境狀態；輸出是動作與停止決策；目標是在成本與風險限制下完成任務。 Input: task and environment state. Output: actions and a stopping decision. Objective: finish under cost and risk constraints.
- 01 把任務轉成可驗證的中間狀態。 Turn the task into verifiable intermediate states.
- 02 依 observation 選擇工具、查詢或內部運算。 Choose tools, queries, or internal computation from observations.
- 03 核對結果，失敗時回復或重規劃。 Verify results, then recover or re-plan after failure.
把漂亮的 chain-of-thought 當成忠實的內部因果紀錄。 Treating a polished chain of thought as a faithful causal record.
比較 agent 時，要固定環境、拆開元件、量測成本，並主動注入失敗。 Compare agents under fixed environments, isolate components, measure cost, and inject failures deliberately.
不用回看頁面的自我檢查Check your recall without looking back
三個回想題Three recall questions
先口頭回答，再展開解析。Answer aloud before opening each explanation.
01 為什麼最終成功率不足以診斷 agent？ Why is final success insufficient to diagnose an agent?
它把規劃、工具、恢復與環境運氣壓成一個數字。 It collapses planning, tools, recovery, and environmental luck into one number.
02 什麼是有用的 failure injection？ What is a useful failure injection?
在可控制的位置加入錯誤或延遲，觀察系統是否偵測並恢復。 Add an error or delay at a controlled point and observe detection and recovery.
03 何時增加推理 token 才合理？ When is a larger reasoning budget justified?
當 ablation 顯示規劃或內部搜尋，而非 observation 品質，才是瓶頸。 When ablations show planning or internal search—not observation quality—is the bottleneck.
把工具帶回你的研究Take the tool back to your research
最後一個任務One final task
挑一篇 agent 論文，把它的成功案例改寫成一個可注入工具錯誤的測試。 Take one agent paper and rewrite its success case as a test with an injectable tool error.