Imagine an assistant watching a live video. It warns about a hazard, but only after the moment has passed. Or it describes a scene accurately while interrupting a user who did not ask for commentary. A response-only score misses the behavior that made either interaction fail.想象一个观看实时视频的助手:它确实提醒了危险,却等到危险已经过去;或者它描述得很准确,却在用户不需要时频繁插话。只给单条回答打分,便看不到造成交互失败的行为。

SVI-Bench treats the user-observable trajectory as the evaluation object. Its 75 expert-authored items span nine scenarios and five dimensions: proactive triggering, silence correctness, latency, response correctness, and delegation with long-horizon memory. Item-specific anchors express what should happen as timestamps, counts, or information points.SVI-Bench 把用户可感知的完整轨迹作为评测对象。75 个专家编写的条目覆盖九类场景和五个维度:主动触发、正确沉默、延迟、回答正确性,以及委派与长程记忆。逐条目标准用时间戳、次数或信息点明确应发生的行为。

Speaking and silence belong together开口与沉默要放在一起评分

A system that stays silent can look excellent on silence correctness while failing to act when needed. A system that speaks constantly can appear responsive while repeatedly interrupting. SVI-Bench couples proactive triggering and silence within each applicable item by taking the lower of their two scores, then combines that component with the other applicable dimensions.始终沉默的系统可能在“正确沉默”上得高分,却错过必须回应的时刻;始终开口的系统可能显得积极,却不断打断用户。SVI-Bench 在每个适用条目内取主动触发与正确沉默两项中的较低分,再与其他适用维度合成条目分数。

The coupling happens before averaging across items. That order matters: it asks whether a system made the right speaking decision in the same situation, rather than letting strength on unrelated examples hide a failure.这一步发生在跨条目平均之前。它检验同一情境中的开口决策是否正确,避免不同样本上的强项掩盖当前条目的失误。

Make time inspectable让时间成为可核查的证据

InteractFlow records synchronized traces of source video, model outputs, and event times. A judge can inspect a response alongside the visual state available when it occurred; the saved trace can be replayed and rescored. The pipeline applies a shared recording and scoring protocol while preserving each system's streaming interface.InteractFlow 同步记录源视频、模型输出和事件时间。判分者能结合回答发生时可见的画面核查它;保存的轨迹也支持复看与重新评分。流程统一录制和评分协议,同时保留各系统原有的流式接口。