ARC-AGI-3: What an Interactive Agentic Benchmark Means for Governed AI Harnessing
ARC-AGI-3 measures how efficiently an AI system acquires a new skill in an unfamiliar, turn-based environment without instructions. Humans solve 100% of its environments; frontier models score below 1%. The report identifies context management in a long agent session — the layer Tvijo AI OS is built around — as the central engineering bottleneck.
On 22 April 2026 the ARC Prize Foundation published the ARC-AGI-3 technical report. It extends the ARC-AGI series (2019, 2024) and ARC-AGI-2 (2025) from static grid puzzles to interactive, turn-based environments. An agent must explore the world, infer its mechanics and reach a win condition that is never explained to it. This is a direct test of exploration, model building, goal-setting and planning — the same pillars that a governed AI harness has to support in a long, verifiable working session.
What ARC-AGI-3 is, in plain terms
- A 64×64 grid with 16 colors, played in turns — difficulty lives in logic, not in reflexes or vision.
- A deliberately small action set: five base actions + Undo + selecting a cell by coordinates.
- The agent is never told the goal or the rules — it must infer the mechanics and the win condition by itself.
- Environments use only Core Knowledge priors (objectness, basic geometry, elementary physics, agency) with no language or cultural symbols, so prior knowledge cannot help.
The report frames four pillars of agentic intelligence it measures: Exploration (actively gathering information), Modeling (building a predictive world model), Goal-Setting (choosing what to target without instructions) and Planning & Execution (path to the goal with course correction).
Intelligence is measured as efficiency, not as solve/no-solve
The methodological core of ARC-AGI-3 is RHAE (Relative Human Action Efficiency): intelligence is not “did it solve”, but “how many actions did it take on first contact”. One scalar — action efficiency — absorbs data, time, compute and risk.
Key scoring mechanics worth understanding:
- Each completed level earns a fraction of the human baseline (upper-median best human run), squared — a power-law penalty that punishes blind brute force.
- A level is capped at 1.15× the human baseline so one exploit cannot inflate the result.
- Levels are linearly weighted (1/15, 2/15 … 5/15 in a five-level environment) so tutorial levels barely move the score.
- An environment cap ensures an agent cannot earn a high score by being efficient on a few early levels while failing the later ones.
This design states plainly that brute force is not intelligence. A system that randomly tries many options earns a low score even if it eventually reaches the goal. That matches how Tvijo already measures its own runs: not just PASS/FAIL, but how many tokens, dollars and steps the path to the answer consumed.
The gap today — and why it matters
Humans vs models (official leaderboard, March 2026):
| Participant | 评分 |
|---|---|
| Human baseline | 100% |
| Anthropic Opus 4.6 (Max) | 0.50% |
| Google Gemini 3.1 Pro Preview | 0.40% |
| OpenAI GPT 5.4 (High) | 0.20% |
| xAI Grok-4.20 (Beta 0309 Reasoning) | 0.10% |
The harness changes everything. The report shows dramatic bimodality on the same frontier model: on environment TR87, Opus 4.6 scores 0.0% without a harness and 97.1% with the university Duke harness; on BP35 it scores 0.0% under both. The authors’ conclusion: frame perception and the API format are not the limiting factor — with the right hand-crafted strategy, frontier models *can* solve these environments. The constraint is the wrapper, not the model’s “vision”.
Competitions. The 30-day preview competition (July–August 2025) was won by StochasticGoose (Tufa Labs) at 12.58% on a CNN + reinforcement-learning approach; second place went to Blind Squirrel (6.71%) on a directed state graph. By the authors’ own description both are, in essence, “informed brute force” — exactly what RHAE is designed to penalise.
ARC Prize 2026 — $2M prize pool, two tracks (ARC-AGI-3 and the final year of ARC-AGI-2), run on Kaggle, open-source required. The report adds that as of March 2026 ARC-AGI-3 is the only unsaturated general agentic-intelligence benchmark.
Why this is about us: the report’s central bottleneck is context management
The report names the core engineering problem directly: context management.
A frame is a 64×64 grid. Holding a naive rolling window of observations exhausts a model’s context budget within a handful of turns. The two external teams that solved all three public environments at human-level efficiency used the same two answers:
- Duke harness — let the model run arbitrary Python to selectively retrieve and transform information from its action history, instead of carrying the whole history in the prompt.
- Symbolica Arcgentica — an orchestrator + subagents architecture: the orchestrator never touches the environment directly, but delegates to specialised subagents that return compressed summaries. Context growth is bounded while the high-level plan survives.
Both patterns — selective history access and “orchestrator + subagents with compression” — are what Tvijo AI OS already does in production: a dispatcher issues isolated work packets, a verifier checks artifacts against a contract, and every run records route, tokens, cost and outcome. We are not inventing ARC-AGI-3 — we are building the layer the report identifies as decisive.
The honest place for a harness: the community leaderboard, not the official one
Precision matters here, and we keep it.
ARC Prize runs two leaderboards. The official one deliberately excludes harnesses: it measures a model’s general intelligence “from a clean slate”, without domain wrappers, and exists to answer “how close are we to AGI”. The community leaderboard is a separate public venue for harness research, with self-reported scores, and the authors explicitly note harness innovation is “not necessarily progress towards AGI, but economically valuable”.
Tvijo AI OS is a harness. Our honest address is therefore the community leaderboard and open harness research — not a claim to AGI. This is not second-class: the report stresses that the best harness ideas eventually flow behind model APIs (chain-of-thought went from an external DeepMind harness around GPT-3 to a first-party feature in o1). Building that layer is exactly what we do — claiming more would be unfair to the reader and to the benchmark itself.
What Tvijo AI OS already has (verified)
| Benchmark requirement / pattern | Tvijo AI OS control plane | 状态 |
|---|---|---|
| 长会话中的上下文管理 | Dispatcher DSH + 隔离工作包，压缩为工件 | 已就位 |
| 编排器 + 子代理 | 跨模型和提供商的角色分离（生成/审计/测试） | 已就位 |
| “不是解决与否，而是成本多少” | 每次运行记录令牌、费用、路由和通过/失败 | 已就位 |
| 防暴力破解保护 | 预算前置门控、并行运行锁定、限制 | 已就位 |
| 可复现性与审计 | 快照清单、SHA-256、日志和可回滚差异 | 已就位 |
| 最终步骤的人工门控 | 发布和部署需人工审批 | 已就位 |
实测内部测试系列：五次运行产生三次通过结果和两次门控拒绝；简报 ≈ $0.0015，带测试修复 ≈ $0.0061，审计 ≈ $0.0031；一次通过结果将退火优化加速了 38–59 倍，同时逐位保持目标不变。这不是“我们已经击败 ARC-AGI-3”——而是证明该任务所需的测量和编排层已就位。
我们针对 ARC-AGI-3 正在做什么
- 在 ARC-AGI-3 工具包上评估就绪度 (Python，按报告规格 ≥1000 fps)：围绕 LRM 的包装器，带选择性操作历史和评分使用的
5n操作预算。 - 两个策略分支 以 Duke（Python 访问历史）和 Arcgentica（编排器 + 子代理）为模型，在公开的 25 个环境集合上进行比较。
- Dispatcher 内的 RHAE 核算：以与当前记录令牌和成本相同的方式记录运行中的代理操作效率，使“知情暴力破解”作为指标可见，而非直觉。
- 在社区排行榜上公开发布 一旦结果存在——自报分数、工具说明、开源（ARC Prize 条件）。
我们不声称在 NDA 下经过验证的参与或分数。就绪度评估和工具研究是当前的事实状态；具体结果将在存在时通过正常证据门控发布。
*编辑注：所有数据来自 ARC-AGI-3 技术报告（ARC Prize 基金会，2026 年 4 月 22 日），反映截至 2026 年 3 月的基准。模型分数随版本变化；转载前请查看 arcprize.org 上的当前排行榜。Tvijo 的内部测试系列测量针对特定配置，不保证性能、节省或任何 ARC-AGI-3 结果。*