ARC-AGI-3: What an Interactive Agentic Benchmark Means for Governed AI Harnessing
ARC-AGI-3 measures how efficiently an AI system acquires a new skill in an unfamiliar, turn-based environment without instructions. Humans solve 100% of its environments; frontier models score below 1%. The report identifies context management in a long agent session — the layer Tvijo AI OS is built around — as the central engineering bottleneck.
On 22 April 2026 the ARC Prize Foundation published the ARC-AGI-3 technical report. It extends the ARC-AGI series (2019, 2024) and ARC-AGI-2 (2025) from static grid puzzles to interactive, turn-based environments. An agent must explore the world, infer its mechanics and reach a win condition that is never explained to it. This is a direct test of exploration, model building, goal-setting and planning — the same pillars that a governed AI harness has to support in a long, verifiable working session.
What ARC-AGI-3 is, in plain terms
- A 64×64 grid with 16 colors, played in turns — difficulty lives in logic, not in reflexes or vision.
- A deliberately small action set: five base actions + Undo + selecting a cell by coordinates.
- The agent is never told the goal or the rules — it must infer the mechanics and the win condition by itself.
- Environments use only Core Knowledge priors (objectness, basic geometry, elementary physics, agency) with no language or cultural symbols, so prior knowledge cannot help.
The report frames four pillars of agentic intelligence it measures: Exploration (actively gathering information), Modeling (building a predictive world model), Goal-Setting (choosing what to target without instructions) and Planning & Execution (path to the goal with course correction).
Intelligence is measured as efficiency, not as solve/no-solve
The methodological core of ARC-AGI-3 is RHAE (Relative Human Action Efficiency): intelligence is not “did it solve”, but “how many actions did it take on first contact”. One scalar — action efficiency — absorbs data, time, compute and risk.
Key scoring mechanics worth understanding:
- Each completed level earns a fraction of the human baseline (upper-median best human run), squared — a power-law penalty that punishes blind brute force.
- A level is capped at 1.15× the human baseline so one exploit cannot inflate the result.
- Levels are linearly weighted (1/15, 2/15 … 5/15 in a five-level environment) so tutorial levels barely move the score.
- An environment cap ensures an agent cannot earn a high score by being efficient on a few early levels while failing the later ones.
This design states plainly that brute force is not intelligence. A system that randomly tries many options earns a low score even if it eventually reaches the goal. That matches how Tvijo already measures its own runs: not just PASS/FAIL, but how many tokens, dollars and steps the path to the answer consumed.
The gap today — and why it matters
Humans vs models (official leaderboard, March 2026):
| Participant | 评分 |
|---|---|
| Human baseline | 100% |
| Anthropic Opus 4.6 (Max) | 0.50% |
| Google Gemini 3.1 Pro Preview | 0.40% |
| OpenAI GPT 5.4 (High) | 0.20% |
| xAI Grok-4.20 (Beta 0309 Reasoning) | 0.10% |
The harness changes everything. The report shows dramatic bimodality on the same frontier model: on environment TR87, Opus 4.6 scores 0.0% without a harness and 97.1% with the university Duke harness; on BP35 it scores 0.0% under both. The authors’ conclusion: frame perception and the API format are not the limiting factor — with the right hand-crafted strategy, frontier models *can* solve these environments. The constraint is the wrapper, not the model’s “vision”.
Competitions. The 30-day preview competition (July–August 2025) was won by StochasticGoose (Tufa Labs) at 12.58% on a CNN + reinforcement-learning approach; second place went to Blind Squirrel (6.71%) on a directed state graph. By the authors’ own description both are, in essence, “informed brute force” — exactly what RHAE is designed to penalise.
ARC Prize 2026 — $2M prize pool, two tracks (ARC-AGI-3 and the final year of ARC-AGI-2), run on Kaggle, open-source required. The report adds that as of March 2026 ARC-AGI-3 is the only unsaturated general agentic-intelligence benchmark.
Why this is about us: the report’s central bottleneck is context management
The report names the core engineering problem directly: context management.
A frame is a 64×64 grid. Holding a naive rolling window of observations exhausts a model’s context budget within a handful of turns. The two external teams that solved all three public environments at human-level efficiency used the same two answers:
- Duke harness — let the model run arbitrary Python to selectively retrieve and transform information from its action history, instead of carrying the whole history in the prompt.
- Symbolica Arcgentica — an orchestrator + subagents architecture: the orchestrator never touches the environment directly, but delegates to specialised subagents that return compressed summaries. Context growth is bounded while the high-level plan survives.
Both patterns — selective history access and “orchestrator + subagents with compression” — are what Tvijo AI OS already does in production: a dispatcher issues isolated work packets, a verifier checks artifacts against a contract, and every run records route, tokens, cost and outcome. We are not inventing ARC-AGI-3 — we are building the layer the report identifies as decisive.
The honest place for a harness: the community leaderboard, not the official one
Precision matters here, and we keep it.
ARC Prize runs two leaderboards. The official one deliberately excludes harnesses: it measures a model’s general intelligence “from a clean slate”, without domain wrappers, and exists to answer “how close are we to AGI”. The community leaderboard is a separate public venue for harness research, with self-reported scores, and the authors explicitly note harness innovation is “not necessarily progress towards AGI, but economically valuable”.
Tvijo AI OS is a harness. Our honest address is therefore the community leaderboard and open harness research — not a claim to AGI. This is not second-class: the report stresses that the best harness ideas eventually flow behind model APIs (chain-of-thought went from an external DeepMind harness around GPT-3 to a first-party feature in o1). Building that layer is exactly what we do — claiming more would be unfair to the reader and to the benchmark itself.
What Tvijo AI OS already has (verified)
| Benchmark requirement / pattern | Tvijo AI OS control plane | ステータス |
|---|---|---|
| Context management in long sessions | Dispatcher DSH + isolated work packets, compression into artifacts | in place |
| Orchestrator + subagents | Role separation (generation/audit/test) across models and providers | in place |
| “Not solve, but how much it cost” | 実行ごとに記録されるトークン、金額、ルート、PASS/FAIL | in place |
| ブルートフォース攻撃への保護 | 事前ゲート予算、並列実行ロック、制限 | in place |
| 再現性と監査 | スナップショットマニフェスト、SHA-256、ジャーナル、リバート可能な差分 | in place |
| 最終ステップでの人間によるゲート | 公開とデプロイは手動承認の後ろに留まる | in place |
測定された内部テストシリーズ: 5回の実行で3件の承認結果と2件のゲート拒否が発生。簡潔版 ≈ $0.0015、修正付きテスト ≈ $0.0061、監査 ≈ $0.0031。承認された1件の結果は、目的をビット単位で保持しつつ、アニーリング最適化を38〜59倍高速化しました。これは「すでにARC-AGI-3を破った」という意味ではなく、タスクに必要な測定・オーケストレーション層が整っているという証拠です。
ARC-AGI-3への取り組み
- ARC-AGI-3ツールキットへの準備状況の評価 (Python、レポート仕様に基づき毎秒1000フレーム以上): 選択的アクション履歴を持つLRMのラッパーと、
5nスコアリングが使用するアクション予算。 - 2つの戦略ブランチ Duke(Pythonによる履歴アクセス)とArcgentica(オーケストレーター+サブエージェント)をモデルに、公開されている25環境のセットで比較。
- ディスパッチャ内のRHAE会計: トークンとコストが今日記録されるのと同じ方法で、実行中のエージェントのアクション効率を記録し、「情報に基づくブルートフォース」を直感ではなく指標として可視化します。
- コミュニティリーダーボードでの公開 結果が出次第 — 自己報告スコア、ハーネス説明、オープンソース(ARC Prizeの条件)。
NDAに基づく検証済みの参加やスコアを主張しているわけではありません。準備状況の評価とハーネス研究が現在の事実上の状態であり、具体的な結果は、存在する場合には通常の証拠ゲートを通じて公開されます。
*編集注: すべての数値はARC-AGI-3テクニカルレポート(ARC Prize Foundation、2026年4月22日)からのもので、2026年3月時点のベンチマークを反映しています。モデルのスコアはリリースによって変わります。転載前にarcprize.orgの現在のリーダーボードを確認してください。Tvijoの内部テストシリーズの測定値は特定の構成を指し、パフォーマンス、節約、またはARC-AGI-3の結果を保証するものではありません。*