A reasoning model can spend a long chain of thought on a question it already knows how to answer. Asking it to be shorter everywhere creates the opposite problem: on an unfamiliar question, those extra steps may be exactly what it needs. CATER starts by asking what the student can solve now.推理模型可能在熟悉的问题上反复展开长思维链。若对所有问题一律要求简短,陌生问题又可能失去必要的探索步骤。CATER 的起点是判断学生模型此刻能解出什么。
For each problem, the current student samples a group of fresh responses. A verifier marks which responses are correct. That pattern is the routing signal: all correct, mixed outcomes, or all failed. The category belongs to the current policy, so it can change as the student learns.每道题都由当前学生模型重新采样一组回答,再由验证器标记正确与否。全对、有对有错、全错三种结果决定训练信号。分类反映的是当前策略的能力,会随模型学习而变化。
Match the signal to the bottleneck让训练信号匹配当前瓶颈
All-correct groups receive concise on-policy self-distillation: a periodically refreshed self-teacher supplies token-level guidance under a brevity instruction. Mixed groups receive correctness-based reinforcement learning, using the successful and unsuccessful responses as a contrast. All-failed groups are temporarily deferred because that group has no successful trajectory to reinforce or safely compress.全对组接受简洁的 on-policy 自蒸馏:定期刷新的自教师在简短作答指令下提供逐 token 指导。有对有错的组通过正确性强化学习利用成败对照。全错组暂时搁置,因为它尚无可强化、可安全压缩的成功轨迹。
The routing is repeated on new student rollouts. A problem that was once all failed can enter the mixed group when successful reasoning appears, then enter the compression group when success becomes consistent. The training curriculum follows demonstrated competence rather than a fixed difficulty label.路由会随着新的学生采样不断重算。原本全错的问题,一旦出现成功轨迹就可以进入混合组;稳定解出后再进入压缩组。训练课程跟随已经表现出的能力,而不是固定难度标签。
Measure both sides of efficiency同时衡量效率与正确性
CATER separates the decision of where to compress from the question of how strongly to combine distillation and reinforcement learning. Its evaluation therefore has two essential axes: answer correctness and generated reasoning tokens. A shorter answer is useful when the model still solves the problem it was meant to solve.CATER 将“哪些题该压缩”与“蒸馏和强化学习如何平衡”分开设计。评估也必须同时看答案正确性和生成的推理 token:只有维持解题能力,缩短回答才有意义。