← All competition runs
Live · closes 2 DecUpdated November 15, 2026

Gemma 4 Developer Agent on Kaggle: Rules, Scoring and Our Run

The Gemma 4 Developer Agent is a Google competition on Kaggle with a $75,000 prize pool that closes on 2 December 2026. You build a coding agent on one Gemma 4 model that fixes bugs in real Python repositories. Each patch is applied in a fresh container and passes or fails on hidden tests, as in SWE-bench.

Competition facts

Host
Google, on Kaggle (Featured)
Prize pool
$75,000 (paper track: $35,000, closes 12 Nov 2026)
Final deadline
2 Dec 2026, 23:59 UTC
Task
A coding agent that fixes bugs in real Python repositories
Score
Share of tasks whose patch passes the hidden tests (pass/fail)
Model
One Gemma 4 base model: 31B, 12B, E4B or E2B, 4-bit quantized
Hardware
4 × NVIDIA L4 (96 GB total), 32,768-token context
Teams
5,772 on the public leaderboard (15 Nov 2026)
Leader
0.29 public (15 Nov 2026)
Bronze-medal line
0.20 public (rank 577, 15 Nov 2026)

Deadline, prize and team count from the Kaggle API; rules from the competition page. Check the official rules before you rely on any of it.

Our run with ResearchForge

2,292 / 5,772
Public rank
0.17
Best public score
39.7%
Top of the field
0.29
Public leader

Medal standing · Bronze line 0.20

The bronze-medal line is 0.20, three tasks in a hundred above our 0.17. Of 24 recorded rounds, 20 failed, most of them on Colab compute rather than on the idea — that is the next thing to fix.

A byte-identical resubmission scored 0.12, then 0.17 — so no single leaderboard number is trusted.

  1. 25 Oct0.05

    Baseline agent on Gemma 4 12B, with the code-graph tools removed

  2. 27 Oct0.15

    Adapted from a public notebook reported at 0.22–0.24, on Gemma 4 31B

  3. 29 Oct0.12

    The exact prompt of a public notebook reported as a stable 0.22

  4. 1 Nov0.17

    Control: the same submission file, byte for byte — a 0.05 swing from noise alone

  5. 7–13 Nov24 runs

    Overnight loop on Colab: 20 rounds failed, mostly dropped or timed-out sessions; best local benchmark 0.40 on 10 tasks, 0.33 on 30

  6. 2 Dec—

    Final deadline

What the record showed

Resubmitting identical code moved the public score by 0.05, so one leaderboard number is treated as noisy evidence, not a verdict.

Read the full write-up

Questions people ask

What is the Gemma 4 Developer Agent competition?

A Google competition on Kaggle where you build a coding agent on Gemma 4 that fixes bugs in real Python repositories. The prize pool is $75,000 and the final deadline is 2 December 2026, 23:59 UTC. A separate paper track ($35,000) closes on 12 November 2026.

How is the Gemma 4 Developer Agent scored?

Like SWE-bench. The agent works on each task in a sandbox, its patch is applied in a fresh container, and hidden tests decide pass or fail. The score is the share of tasks fixed, so the public leader at 0.29 fixes about 29% of the public tasks.

Which models and hardware can a Gemma 4 Developer Agent use?

One base model per submission, chosen from the four official 4-bit Gemma 4 builds (31B, 12B, E4B and E2B), served with vLLM on 4 NVIDIA L4 GPUs with a 32,768-token context. LoRA adapters are allowed inside the submission size limit.

How did ResearchForge do on the Gemma 4 Developer Agent?

Our best public score is 0.17, ranking 2,292 of 5,772 teams on 15 November 2026; the bronze-medal line is 0.20. The most useful finding was about noise: a byte-identical resubmission scored 0.12 and then 0.17. Since then 20 of 24 recorded rounds failed, mostly on Colab sessions dropping or timing out, so compute is the bottleneck to fix.

Sources

Run it on this competition

Kaggle Overnight is open-source autoresearch for Kaggle: it keeps your score level with the best public notebooks, then keeps experimenting on top of them. Give it the competition URL tonight.