成功率只能告诉我们机器人有多少次把任务做完，说不清它是在哪一步出的问题。Mind2Act World 是一个面向机器人操作的诊断式基准：它把「想」、「做」以及两者的配合分成三个视角分别评测，再把每一次失败追溯到具体的环节。本报告介绍基准的设计、评测协议、诊断方法，以及截至 2026 年 11 月 11 日的首批结果。完整的基准、排行榜和任务视频见 mind2act-world.com。
一、问题：一个总分说明不了什么
具身智能要求机器人在一个闭环里推理和行动：推理决定下一个动作，动作的结果又决定下一步怎么想。这背后是三种能力。
- 想（Mind）：记住与任务有关的信息，并在此基础上推理。
- 做（Act）：精确、稳定地执行物理动作。
- 想与做的配合（Mind2Act）：通过环境反馈把两者协调起来。
现有基准大多在同一批任务里同时考推理和执行，失败之后无法判断问题出在「想」还是「做」；把两者分开考的基准，又很少检查物理结果有没有被用来指导下一步决策。而现实中，决策正确的任务可能败在执行上，执行精确的动作也可能是在落实一个错误的决策。
二、设计：三个视角，同一套系统
Mind2Act World 给每种能力一个独立的视角，三个视角各有 10 个任务。
| 视角 | 它问的问题 | 任务如何构造 |
|---|---|---|
| Mind | 行动之前，机器人必须记住或推断出什么？ | 每个操作本身都很简单，但正确的下一步取决于之前观察到的信息。 |
| Act | 机器人能不能把意图变成有效的物理结果？ | 目标说得很明确，成败取决于执行是否精确、时机是否合适。 |
| Mind2Act | 执行的反馈如何改变接下来要做的事？ | 把相互依赖的操作串起来，每一步的物理结果决定下一步做什么。 |
让同一个系统分别经过这三个视角，就能看出它的失败来自「想」、来自「做」，还是来自两者的配合。
规模
| 视角 | 任务 | 难度 | 场景 | 端到端模型用例 | Agent 用例 |
|---|---|---|---|---|---|
| Mind | 10 | 3 | 2 | 3,000 | 300 |
| Act | 10 | 3 | 2 | 3,000 | 300 |
| Mind2Act | 10 | 3 | 2 | 3,000 | 300 |
| 合计 | 30 | 3 | 2 | 9,000 | 900 |
30 个任务来自五类真实工作场景：制造与物流、仪器与工具使用、家居与餐饮服务、陪伴与教育娱乐、装配与物理推理。每个任务有简单、中等、困难三档难度，并在 Clean 和 Random 两种场景下运行，共 180 个配置。此外，基准提供 4,500 条示范数据，用于端到端模型（VLA / WAM）的适配。
12 类能力
| 视角 | 能力 | 任务数 | 难度如何提升 |
|---|---|---|---|
| Mind | 实体记忆 | 2 | 要记住的身份和对应关系更多，检索跨度更长。 |
| Mind | 过程记忆 | 2 | 需要重建的订单或层数更多，并加入干扰项。 |
| Mind | 空间记忆 | 2 | 空间布局变化更快，导航网格更大。 |
| Mind | 反馈规则学习 | 2 | 要从有限反馈中推断的动作或排列空间更大。 |
| Mind | 综合记忆 | 2 | 路线更长且有折返，要跟踪的身份与位置配对更多。 |
| Act | 动态响应 | 4 | 移动目标更多，响应窗口更短，运输约束更紧。 |
| Act | 精确执行 | 6 | 读数和朝向的容差更小，路径和折叠更复杂，物理链条更长。 |
| Mind2Act | 累积进度、前置条件与修复、反馈调整、序列与规划、预测决策 | 10 | 物品或要记住的步骤更多，前置条件更多，对反馈的依赖更紧。 |
三、评测协议
所有模型都由我们在自己的机器上运行，使用隐藏用例和固定的时间预算，并以物理结果计分。
- 两条赛道。端到端模型（VLA / WAM）直接输出机器人动作，每个配置 50 个用例，每个模型共 9,000 个；Agent 通过工具调用来行动，每个配置 5 个用例，共 900 个。两条赛道的预算不同，排行榜也分开。
- 隐藏用例。一个用例固定了任务、难度、随机种子、场景和指令。评测种子不会用于训练，每个用例只运行一次。
- Clean 与 Random 成对。Clean 使用原始场景；Random 保持任务不变，只改变外观：桌面材质（23 种）、光照（6 种光源）和杂物。两者的物体与位姿、机器人、相机、物理参数、种子、指令和计分完全一致，而且每个用例的 Random 外观是固定的，所有模型看到的都一样。
- 物理计分。分数取决于物体最终在哪里、机构是否真的被触发，而不是模型自己的判断。
计分方式
| 指标 | 定义 |
|---|---|
| 用例得分 | 成功记 100 分；其他结果按该任务定义的部分得分计，或记 0 分。 |
| 任务得分 | 有效用例得分的平均值，包含 0 分和部分得分。 |
| 成功率 | 有效用例中得 100 分的比例。 |
| 视角得分 | 该视角 10 个任务得分的平均值。 |
| 总分 | 0.25 × Mind + 0.25 × Act + 0.5 × Mind2Act |
排行榜上的数值是 Clean 和 Random 两种场景的平均值。
四、诊断：分数之外的三件事
除了是否成功，计分器还会记录阶段进度和事件级证据。诊断流程据此为每个被评测的模型生成一张诊断卡，回答三个问题。
- 它走了多远。过程得分统计每个回合达到的里程碑数量。两个成功率同为 0% 的模型，实际进度可以差得很远。
- 它最先在哪里出错。每个失败回合被追溯到第一个无法挽回的环节：感知、推理、执行，或闭环（检查物理结果并据此更新下一步）。
- 为什么，以及什么会有帮助。失败代码指出原因属于 Mind、Act 还是两者的配合，并给出改进方向，例如需要补充哪一类数据。
同样是 0 分，问题可以完全不同。以「用积木搭桥」为例（示意）：模型 A 停在 6 步中的第 1 步，第一次抓取就滑脱，需要的是抓取和抬升的示范数据；模型 B 走到了第 4 步，却在桥墩之前先放了桥面，需要的是会检查前置条件的规划能力。
一张示例诊断卡
下面的数字来自网站上公开的示例卡，对象是一个匿名的端到端 VLA 模型，用于说明诊断卡的结构，不代表任何具名模型的公开结果。
| 视角 | 成功率 | 成功 / 总回合 | 90% 区间 |
|---|---|---|---|
| 总体 | 6.5% | 583 / 9,000 | 6.1–6.9% |
| Mind | 1.9% | 58 / 3,000 | 1.6–2.4% |
| Act | 15.2% | 456 / 3,000 | 14.1–16.3% |
| Mind2Act | 2.3% | 69 / 3,000 | 1.9–2.8% |
在 8,417 个失败回合中，规则层目前已经归因了 2,637 个（31.3%），其余还在等待视频判定，因此下面的分布只代表已归因的部分。
| 最先出错的环节 | 回合数 | 占已归因比例 |
|---|---|---|
| 感知 | 158 | 6.0% |
| 推理 | 1,291 | 49.0% |
| 执行 | 924 | 35.0% |
| 闭环 | 249 | 9.4% |
排在前面的失败原因是：遗忘或记错信息（746 个）、抓取失败或没有抓稳（414 个）、连续精细控制误差（330 个）、计划或步骤顺序错误（289 个）。对这个模型，诊断卡给出的第一条建议是补充「需要把早先的信息或进度带过很多步」的示范数据，用来加强记忆。
五、首批结果
以下为截至 2026 年 11 月 11 日已公布的结果，格式为「得分 / 成功率」，取 Clean 和 Random 的平均值。另有 9 个已完成训练的端到端模型正在等待评测。
端到端模型（VLA / WAM）
| 模型 | 总体 | Mind | Act | Mind2Act |
|---|---|---|---|---|
| KinRT | 8.05 / 7.23% | 5.64 / 4.40% | 18.29 / 14.50% | 4.14 / 2.80% |
| π0.5 | 7.77 / 6.63% | 4.46 / 3.50% | 16.75 / 12.87% | 4.94 / 3.53% |
| G0.5 | 6.64 / 6.48% | 2.80 / 1.93% | 17.74 / 15.20% | 3.02 / 2.30% |
| X-VLA | 5.88 / 6.57% | 1.41 / 1.03% | 20.72 / 18.47% | 0.69 / 0.20% |
Agent
| 模型 | 总体 | Mind | Act | Mind2Act |
|---|---|---|---|---|
| GPT-6 Astra | 29.74 / 34.11% | 53.88 / 52.67% | 39.77 / 38.33% | 12.65 / 11.33% |
| DeepSeek V4.1 Flash | 6.00 / 6.44% | 12.37 / 11.00% | 11.01 / 8.33% | 0.32 / 0.00% |
| Gemini 4 Argon | 4.36 / 4.67% | 4.97 / 4.33% | 12.25 / 9.67% | 0.10 / 0.00% |
两条赛道的用例数量和预算不同，分数不宜直接横向比较。
效率
| 模型 | 单任务成本 | 单任务耗时 | 单次推理 / 调用耗时 |
|---|---|---|---|
| KinRT | — | 4.8 分钟 | 110.9 ms |
| π0.5 | — | 8.6 分钟 | 77.1 ms |
| G0.5 | — | 6.7 分钟 | 170.3 ms |
| GPT-6 Astra | ≈ ¥175.88 | 22.2 分钟 | 11.0 s |
| DeepSeek V4.1 Flash | ≈ ¥8.26 | 96.1 分钟 | 35.6 s |
| Gemini 4 Argon | ≈ ¥7.83 | 18.0 分钟 | 27.8 s |
端到端模型的推理耗时为每个动作块的预测时间，在 RTX PRO 5000 上测得；Agent 的成本和耗时为单次任务运行的平均值。
六、从首批结果里能看到什么
- 端到端模型「做」明显好于「想」。四个模型在 Act 上的成功率是 12.87%–18.47%，而在 Mind 上都不超过 4.40%，在 Mind2Act 上都不超过 3.53%。
- Agent 在「配合」上掉得最多。三个 Agent 的 Mind2Act 成功率都是各自三个视角里最低的；成绩最好的 GPT-6 Astra 在 Mind 上达到 52.67%，到 Mind2Act 只有 11.33%，另外两个为 0%。
- 单看总分会漏掉这些差异。四个端到端模型的总体成功率都在 6.5%–7.2% 之间，但 X-VLA 的 Act 最高（18.47%）、Mind2Act 最低（0.20%），和 π0.5 的能力结构并不一样。
- 想清楚的成本目前很高。成绩最好的 Agent 平均每个任务约 ¥175.88、22.2 分钟，每次调用 11.0 秒；端到端模型每个动作块的预测在 77–170 毫秒。
样本还少，特别是 Agent 赛道只有三个模型，这些观察会随着更多模型完成评测而更新。
七、现状与下一步
- 评测和诊断卡已经可用；把诊断结果反馈回模型（为端到端模型定向补数据，为 Agent 建立经验库），再评测一轮，这一步仍在开发中。
- 完整技术报告、评测代码与任务资产、数据集（任务配置与示范）即将发布。
- 想为自己的模型生成诊断卡的团队，可以通过 hello@velabot.ai 联系我们。
A success rate tells you how often a robot finishes a task. It does not tell you where things went wrong. Mind2Act World is a diagnostic benchmark for robotic manipulation: it evaluates thinking, acting and the coordination between them in three separate views, then traces each failure to a specific step. This report covers the benchmark design, the evaluation protocol, the diagnosis method, and first results as of 11 November 2026. The full benchmark, leaderboard and task videos are at mind2act-world.com.
1. The problem: one score explains very little
Embodied intelligence requires a robot to reason and act in a closed loop. Reasoning guides each action, and the outcome of that action shapes the next decision. Three capabilities sit underneath.
- Mind: remembering task-relevant information and reasoning over it.
- Act: executing physical actions precisely and reliably.
- Mind-action composition (Mind2Act): coordinating the two through feedback from the environment.
Most existing benchmarks test reasoning and execution inside the same tasks, so a failure cannot be traced to Mind or to Act. Those that test them separately rarely check whether physical outcomes inform the next decision. In practice a correct decision can still fail in execution, and precise execution can carry out the wrong decision.
2. Design: three views of the same system
Mind2Act World gives each capability its own view, with ten tasks per view.
| View | Question it asks | How tasks are built |
|---|---|---|
| Mind | What must the robot remember or infer before it acts? | Each operation is simple, but the correct next step depends on what was observed earlier. |
| Act | Can the robot turn the intended action into a valid physical outcome? | The goal is stated explicitly, and success depends on precise, well-timed execution. |
| Mind2Act | How does execution feedback change what happens next? | Dependent operations are chained, so the physical outcome of each step decides the next one. |
Comparing one system across the three views shows whether its failures come from Mind, from Act, or from their composition.
Scale
| View | Tasks | Levels | Settings | End-to-end cases | Agent cases |
|---|---|---|---|---|---|
| Mind | 10 | 3 | 2 | 3,000 | 300 |
| Act | 10 | 3 | 2 | 3,000 | 300 |
| Mind2Act | 10 | 3 | 2 | 3,000 | 300 |
| Total | 30 | 3 | 2 | 9,000 | 900 |
The 30 tasks come from real workflows in five application domains: manufacturing and logistics, instruments and tool use, home and food service, companion and edutainment, and assembly and physical reasoning. Each task has Easy, Medium and Hard levels and runs in Clean and Random scenes, for 180 configurations. The benchmark also provides 4,500 demonstrations for adapting end-to-end (VLA / WAM) models.
Twelve capabilities
| View | Capability | Tasks | What increases with difficulty |
|---|---|---|---|
| Mind | Entity memory | 2 | More identities and associations to retain, longer retrieval spans. |
| Mind | Process memory | 2 | More accumulated orders or layers to reconstruct, with distractors. |
| Mind | Spatial memory | 2 | Faster changes to remembered layouts and larger navigation grids. |
| Mind | Feedback-rule learning | 2 | Larger action or permutation spaces to infer from limited feedback. |
| Mind | Integrated memory | 2 | Longer routes with revisits and more identity–position pairs to track. |
| Act | Dynamic response | 4 | More moving targets, shorter response windows, tighter transport constraints. |
| Act | Precise execution | 6 | Tighter reading and heading tolerances, more complex paths and folds, longer physical chains. |
| Mind2Act | Cumulative progress, prerequisites and repair, feedback adjustment, sequence and plan, predictive decision | 10 | More items or remembered steps, more prerequisites, tighter dependence on feedback. |
3. Evaluation protocol
We run every model ourselves, on hidden cases with fixed time budgets, and score the physical result.
- Two tracks. End-to-end models (VLA / WAM) output robot actions directly and get 50 cases per configuration, 9,000 per model. Agents act through tools and get 5 cases per configuration, 900 in total. The tracks have different budgets and separate leaderboards.
- Hidden cases. A case fixes the task, level, seed, scene and instruction. Evaluation seeds are never used for training, and each case is run once.
- Paired Clean and Random. Clean uses the original scene. Random keeps the task the same and changes only its look: table material (23 options), lighting (6 light types) and clutter. Objects and poses, robot, camera, physics, seed, instruction and scoring are identical across the pair, and the Random look of a case is fixed so every model sees the same one.
- Physical scoring. Scores come from where objects end up and what the mechanisms register, not from the model's own judgment.
Scoring
| Measure | Definition |
|---|---|
| Case score | A successful case scores 100. Other outcomes get the partial credit defined for that task, or zero. |
| Task score | Mean case score over valid cases, including zeros and partial credit. |
| Success rate | Share of valid cases that score 100. |
| View score | Mean over the ten tasks in that view. |
| Overall score | 0.25 × Mind + 0.25 × Act + 0.5 × Mind2Act |
Leaderboard values are the mean of the Clean and Random settings.
4. Diagnosis: three things beyond the score
Beyond success, the scorers record stage progress and event-level evidence. From these, the diagnosis harness produces a diagnosis card for each evaluated model, answering three questions.
- How far it gets. A process score counts the milestones each episode reaches. Two models at 0% success can differ widely in actual progress.
- Where it first fails. Each failed episode is traced to the first step it could not recover from: perceive, reason, execute, or close the loop (check the physical outcome and update the next step from it).
- Why, and what would help. A failure code places the cause in Mind, Act or their composition, and points to a direction, such as the kind of data to add.
The same zero can hide different problems. Take building a bridge from blocks (illustrative): Model A stops at step 1 of 6 because the first grasp slips, and needs grasp-and-lift demonstrations. Model B reaches step 4 but places the deck before the piers, and needs planning that checks prerequisites.
An example diagnosis card
The numbers below come from the example card published on the site for an anonymized end-to-end VLA. It illustrates how a card is laid out and is not a published result for any named model.
| View | Success | Successes / episodes | 90% interval |
|---|---|---|---|
| Overall | 6.5% | 583 / 9,000 | 6.1–6.9% |
| Mind | 1.9% | 58 / 3,000 | 1.6–2.4% |
| Act | 15.2% | 456 / 3,000 | 14.1–16.3% |
| Mind2Act | 2.3% | 69 / 3,000 | 1.9–2.8% |
Of 8,417 failed episodes, the rule layer has attributed 2,637 so far (31.3%). The rest await the video judge, so the breakdown below covers only the attributed share.
| First step to fail | Episodes | Share of attributed |
|---|---|---|
| Perceive | 158 | 6.0% |
| Reason | 1,291 | 49.0% |
| Execute | 924 | 35.0% |
| Close the loop | 249 | 9.4% |
The leading causes are forgotten or misremembered information (746), a grasp that failed or was not held (414), continuous fine-control error (330), and a wrong plan or step order (289). For this model, the card's first recommendation is demonstrations in which earlier information or progress must be carried through many steps, to strengthen memory.
5. First results
Results published as of 11 November 2026, shown as score / success rate, averaged over Clean and Random. Nine more trained end-to-end models are awaiting evaluation.
End-to-end models (VLA / WAM)
| Model | Overall | Mind | Act | Mind2Act |
|---|---|---|---|---|
| KinRT | 8.05 / 7.23% | 5.64 / 4.40% | 18.29 / 14.50% | 4.14 / 2.80% |
| π0.5 | 7.77 / 6.63% | 4.46 / 3.50% | 16.75 / 12.87% | 4.94 / 3.53% |
| G0.5 | 6.64 / 6.48% | 2.80 / 1.93% | 17.74 / 15.20% | 3.02 / 2.30% |
| X-VLA | 5.88 / 6.57% | 1.41 / 1.03% | 20.72 / 18.47% | 0.69 / 0.20% |
Agents
| Model | Overall | Mind | Act | Mind2Act |
|---|---|---|---|---|
| GPT-6 Astra | 29.74 / 34.11% | 53.88 / 52.67% | 39.77 / 38.33% | 12.65 / 11.33% |
| DeepSeek V4.1 Flash | 6.00 / 6.44% | 12.37 / 11.00% | 11.01 / 8.33% | 0.32 / 0.00% |
| Gemini 4 Argon | 4.36 / 4.67% | 4.97 / 4.33% | 12.25 / 9.67% | 0.10 / 0.00% |
The two tracks differ in case counts and budgets, so their scores should not be compared directly.
Efficiency
| Model | Cost per task | Time per task | Time per prediction / call |
|---|---|---|---|
| KinRT | — | 4.8 min | 110.9 ms |
| π0.5 | — | 8.6 min | 77.1 ms |
| G0.5 | — | 6.7 min | 170.3 ms |
| GPT-6 Astra | ≈ ¥175.88 | 22.2 min | 11.0 s |
| DeepSeek V4.1 Flash | ≈ ¥8.26 | 96.1 min | 35.6 s |
| Gemini 4 Argon | ≈ ¥7.83 | 18.0 min | 27.8 s |
For end-to-end models, prediction time is per action chunk, measured on an RTX PRO 5000. For agents, cost and time are averages for one task run.
6. What the first results show
- End-to-end models act far better than they think. The four models reach 12.87%–18.47% success on Act, but none exceeds 4.40% on Mind or 3.53% on Mind2Act.
- Agents lose the most on composition. For all three agents, Mind2Act is the lowest of their three views. The strongest, GPT-6 Astra, reaches 52.67% on Mind and 11.33% on Mind2Act. The other two are at 0%.
- The overall number hides these differences. All four end-to-end models sit between 6.5% and 7.2% overall success, yet X-VLA has the highest Act (18.47%) and the lowest Mind2Act (0.20%), a different profile from π0.5.
- Good reasoning is currently expensive. The strongest agent averages about ¥175.88 and 22.2 minutes per task, at 11.0 s per call. End-to-end models predict an action chunk in 77–170 ms.
The sample is still small, with only three models on the agent track, and these observations will be updated as more models are evaluated.
7. Status and next steps
- Evaluation and diagnosis cards are available today. Feeding a diagnosis back into the model (targeted data for end-to-end models, an experience library for agents) and evaluating again is still in development.
- The full technical report, evaluation code and task assets, and the dataset of task configurations and demonstrations are coming soon.
- Teams that want a diagnosis card for their own model can reach us at hello@velabot.ai.