WRITING / FIELD NOTES写作 / 研究手记
A place to think in public.把思考写出来,供人检验。
Short, concrete notes from generative modeling, multimodal learning, agents, and training systems. Each essay starts with a design question.来自生成模型、多模态学习、智能体与训练系统的短文,从一个具体的设计问题开始。
When should a model stop thinking?模型什么时候该少想一步?
CATER routes each question by the student's current success: develop uncertain capability with RL, then compress reliable reasoning through concise self-distillation.CATER 依据学生当前解题情况分配训练信号:先用强化学习巩固不稳定的能力,再通过简洁自蒸馏压缩稳定成功的推理。
When the teacher disagrees with a correct answer教师与正确解答意见相左时
SP-OPD projects the teacher target onto a constraint supplied by a verifier, then carries that target through both distillation loss terms.SP-OPD 将教师分布投影到验证器给出的约束内,并让两项蒸馏损失使用同一个目标分布。
The unit of video interaction is a trajectory流式视频评测,该看整段交互
SVI-Bench scores speaking and silence together, while InteractFlow preserves the video state and timing behind every response.SVI-Bench 联合评估开口与沉默;InteractFlow 保留每次回答对应的视频画面和时间。
When does a color become the car's?颜色,究竟属于谁?
A diffusion model can draw every requested object and color, then assign the colors to the wrong objects. Counterfactual prompt switches offer a way to study when that assignment changes.扩散模型可能画对对象和颜色,却配错归属。反事实提示词切换提供了研究归属何时还能改变的方法。
A vector is not yet a token向量进了模型,不代表模型读懂了
Projecting fixed multimodal features to the right hidden size solves a wiring problem. Readability needs an interface—and controls that show what the backbone actually contributes.把固定多模态特征投影到正确维度,只解决了接线问题。可读性需要更好的接口,也需要辨明骨干贡献的对照实验。
A hard question is not necessarily a hard search难题,不一定需要难搜索
Multi-hop questions can be solved with one search. What matters for agent training is the path a model must take to find the evidence.多跳问题也可能一次检索就得到答案。训练搜索智能体时,更重要的是模型找到证据的路径。
A search agent needs a ledger, not a longer transcript搜索智能体需要一本账,而非更长的对话
A persistent record of evidence, candidates, and missing constraints helps the agent carry useful state through a long run.持久记录证据、候选与缺失约束,才能让智能体在长程任务里带着有效状态前进。
The data contract is part of the model数据契约,也是模型的一部分
Supervision masks, packed sequences, tool turns, and checkpoint lineage decide which behavior the model can actually learn.监督掩码、序列打包、工具轮次和模型权重血缘,决定模型究竟学到了什么。