先看這個問題為什麼值得解Why this problem is worth solving
情境先行Start with the situation
餐廳宣稱出餐快三倍，但先把半成品做好、一次只接大團客；這跟模型 benchmark 的 batch trick 很像。 A restaurant claims 3× faster service after pre-cooking and accepting only large groups—much like a benchmark batch trick.
系統成本會在 precompute、memory、hardware 與 quality 之間移動，不會憑空消失。 System cost shifts among precomputation, memory, hardware, and quality; it does not vanish.
先備知識Prerequisite
- 理解 latency 與 throughput 不同 Know the difference between latency and throughput
讀完你能做到After this path
- 建立端到端成本表 Build an end-to-end cost ledger
- 辨識不公平的硬體與批次比較 Spot unfair hardware and batch comparisons
- 讀懂 quality–cost Pareto front Read a quality–cost Pareto frontier
閱讀框架Concept frame
- 核心問題Question
- 一個方法快 3 倍，可能只是把成本藏到哪裡？ If a method is three times faster, where might the cost be hidden?
- 對照軸Comparison axis
- 服務的 batch 壓力 Serving batch pressure
- 判讀方式Evidence to inspect
- 並排閱讀三種相對情境，再回到論文檢查消融實驗、反事實測試與適用範圍。Read three relative scenarios side by side, then inspect the paper's ablations, counterfactual tests, and scope statements.
回答後查看解析Answer, then view the explanation
你的預測Make a prediction
比較 inference latency 時，最需要一起固定哪組條件？ Which conditions matter most when comparing inference latency?
看現象，不急著講故事Observe before narrating
三個診斷訊號Three diagnostic signals
這些是待檢查的診斷假說。下一步用條件比較、消融實驗或反事實測試排除其他解釋。These are diagnostic hypotheses to test. Use condition comparisons, ablations, or counterfactual tests to rule out alternatives.
一次看完三種情境Read three scenarios side by side
三種情境對照Three-scenario comparison
哪個最佳化會在低 batch 與高 batch 得到相反排名？ Which optimization might rank oppositely at low and high batch?
對照軸：Comparison axis: 服務的 batch 壓力 Serving batch pressure
單請求 latency、啟動成本與 tail latency 優先。 Prioritize single-request, startup, and tail latency.
同時報 latency distribution 與 throughput。 Report latency distributions and throughput together.
排程與 memory bandwidth 可能主導模型 FLOPs。 Scheduling and memory bandwidth may dominate model FLOPs.
從直覺到機制From intuition to mechanism
把機制講清楚Explain the mechanism
系統論文像搬家：箱子沒有變少，只是有人把它放進倉庫、卡車或你的客廳。 Systems work is like moving house: boxes do not disappear; they move into storage, trucks, or your living room.
輸入是指定的工作負載與服務限制；輸出是品質與成本向量；目標是改善 Pareto 前緣，而非只追一個數字。 Input: a specified workload and serving constraints. Output: a quality-and-cost vector. Objective: improve the Pareto frontier, not one number.
- 01 定義 workload、hardware 與品質約束。 Define workload, hardware, and quality constraints.
- 02 定位 compute、memory、communication 或 scheduling 瓶頸。 Locate compute, memory, communication, or scheduling bottlenecks.
- 03 報告端到端成本與 crossover point。 Report end-to-end cost and crossover points.
用不同 precision、batch 或輸出長度製造 speedup。 Creating a speedup through different precision, batch, or output length.
可信的效率 claim 必須說明 workload、硬體、品質守恆與成本轉移。 Credible efficiency claims specify workload, hardware, quality preservation, and cost transfer.
不用回看頁面的自我檢查Check your recall without looking back
三個回想題Three recall questions
先口頭回答，再展開解析。Answer aloud before opening each explanation.
01 latency 與 throughput 為何可能反向？ Why can latency and throughput move in opposite directions?
batching 提高設備利用率，但每個請求可能等待更久。 Batching improves utilization while each request may wait longer.
02 什麼是 crossover point？ What is a crossover point?
兩方法在 workload 規模改變時，成本排名交換的位置。 The workload scale at which two methods swap cost rankings.
03 Pareto improvement 代表什麼？ What is a Pareto improvement?
至少一項品質／成本變好，其他受關注項目不變差。 At least one quality/cost dimension improves without worsening the others of interest.
把工具帶回你的研究Take the tool back to your research
最後一個任務One final task
把一篇效率論文的主表重畫成 quality–latency–memory 三軸 ledger。 Redraw one efficiency paper's main table as a quality–latency–memory ledger.