ARC-AGI-3 is the hardest benchmark in AI right now. François Chollet, the creator of the original ARC dataset, designed it to test one specific thing: fluid intelligence — the ability to learn new skills on the fly, in situations you've never seen before. Not pattern recognition. Not memorization. Actual generalization.
The competition runs until November 2026. Total prize pool: $850,000, with a $700,000 bonus if anyone hits 100% accuracy. As of August 2026, nobody has cracked it. The current public leaderboard leader is at a fraction of that.
My current position: Rank #203 out of ~2,226 teams. Bronze medal zone. Score: 1.21. This is the story of how I got there — and how ResearchForge tracked every step of the way.
What ARC-AGI-3 actually is
Unlike ARC-AGI-1 and 2 (which showed static grid puzzles), ARC-AGI-3 is an interactive environment. Your agent must explore, take actions, observe consequences, and adapt — all within a hidden, never-before-seen environment. Each game is scored 0–100% based on how well your agent matched human-level performance (both task completion and efficiency of actions taken).
Submissions run as notebooks on Kaggle with RTX 6000 GPUs (48GB VRAM), no internet, up to 9 hours runtime. The agent must operate fully autonomously.
The baseline: probe-based exploration (score 0.08)
My initial approach was a probe-based agent: exhaustively try actions at each position, observe which ones change the environment state, and build a local model of the rules. Scored 0.08 on the public leaderboard.
The probe approach had a fundamental flaw: it treated each grid cell independently. ARC-AGI-3 environments have global structure — actions in one region affect constraints in another. Local probing misses this entirely.
Literature search: what do we know about fluid intelligence in AI?
Hypothesis generation and the key insight
hyp-002 was the one that changed everything. The graph-frontier paper showed that treating an unknown environment as a graph — where nodes are observed states and edges are valid actions — and then expanding the exploration frontier toward nodes with the highest information gain dramatically outperformed blind search.
Combined with Qwen3.6-27B (quantized FP8, running on the RTX 6000's 48GB VRAM via vLLM) as the action selector, the graph-frontier approach gave the agent global awareness of the environment structure while the LLM provided semantic reasoning about what actions made sense at the frontier.
The experiment chain
exp-001FAILhyp-001: Object-centric decomposition — overhead too high0.0621-0.018exp-002WINNERhyp-002: Graph-frontier exploration (base)0.8814+0.801exp-003PASShyp-002 + hyp-003: Graph-frontier + Qwen3-14B action selection1.0221+0.942exp-004WINNERhyp-002 + hyp-003 v2: Qwen3.6-27B-FP8 via vLLM (RTX6000)1.2103+1.130What each experiment revealed
exp-001 failed. Object-centric decomposition added too much computational overhead in the 9-hour submission window — the agent couldn't process enough environments to score well. Rejected with reason: timeout constraint.
exp-002 was the breakthrough. Graph-frontier alone (no LLM, just graph-guided heuristic search) jumped the score from 0.08 to 0.88 — a 10× improvement. The graph representation gave the agent global structure awareness for the first time.
exp-003 added Qwen3-14B as the action selector at the frontier. Score: 1.02. The LLM's semantic reasoning about which frontier nodes to expand first was meaningfully better than the heuristic.
exp-004 — the current best — scales the LLM to Qwen3.6-27B at FP8 precision. At 27B parameters in FP8, the model fits in the RTX 6000's 48GB VRAM with room for the vLLM KV cache. The larger model's reasoning quality showed up directly in the score: 1.21.
Current standing and what's next
The competition closes November 2, 2026. Three months remain. The hypotheses currently queued in ResearchForge for the next run:
- ▸hyp-005: Learned frontier priority function (train a small policy net on exp-004's rollouts)
- ▸hyp-006: Multi-agent setup — two agents explore the same environment with different random seeds, merge their graphs
- ▸hyp-007: Constitutional constraints — add explicit symmetry and invariance rules as hard constraints on the graph expansion
This post will be updated as the competition progresses. Follow Forger Labs HQ on YouTube for live experiment breakdowns.