ARC-AGI-3: Was ein interaktiver agentischer Benchmark für gesteuerte KI-Nutzung bedeutet

ARC-AGI-3 misst, wie effizient ein KI-System eine neue Fähigkeit in einer unbekannten, rundenbasierten Umgebung ohne Anweisungen erwirbt. Menschen lösen 100 % seiner Umgebungen; Spitzenmodelle erzielen unter 1 %. Der Bericht identifiziert das Kontextmanagement in einer langen Agentensitzung — die Ebene, um die Tvijo AI OS aufgebaut ist — als den zentralen technischen Engpass.

Am 22. April 2026 veröffentlichte die ARC Prize Foundation den technischen Bericht zu ARC-AGI-3. Er erweitert die ARC-AGI-Serie (2019, 2024) und ARC-AGI-2 (2025) von statischen Gitterrätseln zu interaktiven, rundenbasierten Umgebungen. Ein Agent muss die Welt erkunden, ihre Mechanik ableiten und einen Siegzustand erreichen, der ihm nie erklärt wird. Dies ist ein direkter Test von Erkundung, Modellbildung, Zielsetzung und Planung — dieselben Säulen, die eine gesteuerte KI-Nutzung in einer langen, überprüfbaren Arbeitssitzung unterstützen muss.

Was ARC-AGI-3 in einfachen Worten ist

• A 64×64-Raster mit 16 Farben, in Runden gespielt — die Schwierigkeit liegt in der Logik, nicht in Reflexen oder Sicht.
• Ein bewusst kleiner Aktionssatz: fünf Basisaktionen + Rückgängig + Auswählen einer Zelle per Koordinaten.
• Dem Agenten wird das Ziel oder die Regeln nie mitgeteilt — er muss die Mechanik und den Siegzustand selbst ableiten.
• Umgebungen nutzen nur Core-Knowledge-Priors (Objektbezogenheit, grundlegende Geometrie, elementare Physik, Handlungsfähigkeit) mit keinerlei Sprach- oder Kultursymbolen, sodass Vorwissen nicht helfen kann.

Der Bericht benennt vier Säulen agentischer Intelligenz, die er misst: Erkundung (aktives Sammeln von Informationen), Modellierung (building a predictive world model), Goal-Setting (choosing what to target without instructions) and Planning & Execution (path to the goal with course correction).

Intelligence is measured as efficiency, not as solve/no-solve

The methodological core of ARC-AGI-3 is RHAE (Relative Human Action Efficiency): intelligence is not “did it solve”, but “how many actions did it take on first contact”. One scalar — action efficiency — absorbs data, time, compute and risk.

Key scoring mechanics worth understanding:

• Each completed level earns a fraction of the human baseline (upper-median best human run), squared — a power-law penalty that punishes blind brute force.
• A level is capped at 1.15× the human baseline so one exploit cannot inflate the result.
• Levels are linearly weighted (1/15, 2/15 … 5/15 in a five-level environment) so tutorial levels barely move the score.
• An environment cap ensures an agent cannot earn a high score by being efficient on a few early levels while failing the later ones.

This design states plainly that brute force is not intelligence. A system that randomly tries many options earns a low score even if it eventually reaches the goal. That matches how Tvijo already measures its own runs: not just PASS/FAIL, but how many tokens, dollars and steps the path to the answer consumed.

The gap today — and why it matters

Humans vs models (official leaderboard, March 2026):

The harness changes everything. The report shows dramatic bimodality on the same frontier model: on environment TR87, Opus 4.6 scores 0.0% without a harness and 97.1% with the university Duke harness; on BP35 it scores 0.0% under both. The authors’ conclusion: frame perception and the API format are not the limiting factor — with the right hand-crafted strategy, frontier models *can* solve these environments. The constraint is the wrapper, not the model’s “vision”.

Competitions. The 30-day preview competition (July–August 2025) was won by StochasticGoose (Tufa Labs) at 12.58% on a CNN + reinforcement-learning approach; second place went to Blind Squirrel (6.71%) on a directed state graph. By the authors’ own description both are, in essence, “informed brute force” — exactly what RHAE is designed to penalise.

ARC Prize 2026 — $2M prize pool, two tracks (ARC-AGI-3 and the final year of ARC-AGI-2), run on Kaggle, open-source required. The report adds that as of March 2026 ARC-AGI-3 is the only unsaturated general agentic-intelligence benchmark.

Why this is about us: the report’s central bottleneck is context management

The report names the core engineering problem directly: context management.

A frame is a 64×64 grid. Holding a naive rolling window of observations exhausts a model’s context budget within a handful of turns. The two external teams that solved all three public environments at human-level efficiency used the same two answers:

• Duke harness — let the model run arbitrary Python to selectively retrieve and transform information from its action history, instead of carrying the whole history in the prompt.
• Symbolica Arcgentica — an orchestrator + subagents architecture: the orchestrator never touches the environment directly, but delegates to specialised subagents that return compressed summaries. Context growth is bounded while the high-level plan survives.

Both patterns — selective history access and “orchestrator + subagents with compression” — are what Tvijo AI OS already does in production: a dispatcher issues isolated work packets, a verifier checks artifacts against a contract, and every run records route, tokens, cost and outcome. We are not inventing ARC-AGI-3 — we are building the layer the report identifies as decisive.

The honest place for a harness: the community leaderboard, not the official one

Precision matters here, and we keep it.

ARC Prize runs two leaderboards. The official one deliberately excludes harnesses: it measures a model’s general intelligence “from a clean slate”, without domain wrappers, and exists to answer “how close are we to AGI”. The community leaderboard is a separate public venue for harness research, with self-reported scores, and the authors explicitly note harness innovation is “not necessarily progress towards AGI, but economically valuable”.

Tvijo AI OS is a harness. Our honest address is therefore the community leaderboard and open harness research — not a claim to AGI. This is not second-class: the report stresses that the best harness ideas eventually flow behind model APIs (chain-of-thought went from an external DeepMind harness around GPT-3 to a first-party feature in o1). Building that layer is exactly what we do — claiming more would be unfair to the reader and to the benchmark itself.

What Tvijo AI OS already has (verified)

Measured internal test series: five runs produced three accepted results and two gate rejections; brief ≈ $0.0015, fix-with-tests ≈ $0.0061, audit ≈ $0.0031; one accepted result accelerated an anneal optimisation by 38–59× while preserving the objective bit-for-bit. This is not “we already beat ARC-AGI-3” — it is evidence that the measurement and orchestration layer needed for the task is in place.

What we are doing about ARC-AGI-3

1. Assessing readiness on the ARC-AGI-3 toolkit (Python, ≥1000 fps per the report’s spec): a wrapper around an LRM with selective action history and the 5n action budget the scoring uses.
2. Two strategy branches modelled on Duke (Python access to history) and Arcgentica (orchestrator + subagents), compared on the public set of 25 environments.
3. RHAE accounting inside the dispatcher: record agent action efficiency in a run the same way tokens and cost are recorded today, so “informed brute force” is visible as a metric rather than intuition.
4. Open publication on the community leaderboard once results exist — self-reported score, harness description, open-source (the ARC Prize condition).

We are not claiming verified participation or scores under NDA. Readiness assessment and harness research are the current, factual state; concrete results will be published through the normal evidence gates when they exist.

*Editorial note: all figures are from the ARC-AGI-3 Technical Report (ARC Prize Foundation, 22 April 2026) and reflect the benchmark as of March 2026. Model scores change with releases; check the current leaderboard on arcprize.org before reprinting. Tvijo’s internal test-series measurements refer to a specific configuration and are not a guarantee of performance, savings or any ARC-AGI-3 result.*