← All posts
ROGII Wellbore Geology Prediction · Kaggle · $50,000 prize pool
Case Study

RMSE 15 → 6: How I Used an AI Research Loop on a $50k Kaggle Competition

How systematically grounding experiments in literature cut prediction error by 60% on the ROGII Wellbore Geology challenge.

15.2 → 6.1
RMSE
60%
Improvement
~1,700 / 6,125
Rank
9 (3 rejected)
Experiments
Manas Joshi·August 6, 2026·12 min read

In May 2026, ROGII — a geoscience software company — launched a Kaggle competition asking participants to predict the geology encountered along horizontal wellbores. The dataset: drilling measurements from real wells. The target: tvt, a continuous value representing the rock layer position. The prize pool: $50,000. The metric: RMSE — lower is better.

I started with a naïve gradient-boosted baseline. RMSE: 15.2. Rank: somewhere past 4,000. I had 6,125 teams ahead of me.

What I didn't have was a systematic way to know which ideas were worth trying — or why the ones I had tried didn't work. That's when I ran ResearchForge for the first time.

The problem: drilling underground without a map

A horizontal wellbore is drilled sideways through rock layers that are, at best, partially understood from seismic surveys. The drill bit needs to stay inside the "pay zone" — the productive rock layer — but the layers aren't flat. They dip, fault, and vary in thickness. Predicting where the bit is relative to the formation geology is the entire problem.

The input features were logging-while-drilling (LWD) measurements: gamma ray, resistivity, density, neutron porosity, and positional data. The target tvt (true vertical thickness) represents the geological depth offset from the formation top.

🔍
Why this is hard: The same LWD readings can correspond to completely different geological positions depending on the well trajectory, the local formation dip, and the rock facies. Models that memorize sensor patterns overfit. Models that capture the physical relationship generalize.

Step 1: Let ResearchForge search the literature

Before writing a single line of model code, I ran a literature search:

bash
$researchforge research search "wellbore geology prediction LWD formation evaluation"
🔍 Searching arXiv...
📚 Retrieved 847 candidate papers
🏆 Top results by relevance:
[0.94] Sequence-based lithology prediction using deep learning
[0.91] Transfer learning for geophysical core analysis
[0.89] Multi-task learning for formation evaluation from LWD logs
[0.87] Attention mechanisms in subsurface sequential data
[0.85] Self-supervised pre-training on well log representations
[0.82] Uncertainty quantification in geosteering decisions
✓ Stored 20 papers in local knowledge base.

Three papers immediately stood out. The sequence-based paper showed that treating LWD logs as a time series — with the drill bit position as the "sequence position" — significantly outperformed feature-engineered tabular models. The multi-task paper showed that jointly predicting multiple geological properties (not just TVT) as auxiliary tasks improved the primary target. The attention paper suggested that long-range dependencies in the log sequence mattered.

None of these ideas had appeared in my initial search. The winning idea, it turned out, came from the first paper — one I'd never have found manually in a reasonable amount of time.

Step 2: Freeze the baseline

bash
$researchforge baseline run --metric rmse --higher-is-better false
⏳ Running baseline (train.py)...
📊 val_rmse = 15.2441
🔒 Baseline frozen at commit abc1234
Metric: rmse = 15.2441
Script: train.py
Time: 2026-06-03 09:14:22 UTC

This is the step most people skip. Before running any experiment, ResearchForge commits the baseline metric cryptographically. Every subsequent improvement is measured against this exact number — not against whatever I last ran, not against a "pretty good" run from yesterday.

Step 3: Generate and run hypotheses

bash
$researchforge hypotheses generate
📝 Generating hypotheses from 20 stored papers...
hyp-001: Treat LWD sequence as temporal → use LSTM/GRU encoder
Evidence: papers [0.94], [0.87] (attention on sequences)
hyp-002: Add depth-normalized positional encoding to feature set
Evidence: paper [0.94] (drill position as sequence index)
hyp-003: Multi-task: jointly predict gamma + resistivity as aux targets
Evidence: paper [0.89] (multi-task formation evaluation)
hyp-004: Self-supervised pre-training on unlabeled well logs
Evidence: paper [0.85]
hyp-005: Per-well normalization before feature engineering
Evidence: domain knowledge + paper [0.82]
hyp-006: Rolling window features at 3 scales (5, 20, 50 steps)
Evidence: paper [0.87] (multi-scale attention)
hyp-007: Ensemble GBM + sequence model, weight by well type
Evidence: blending strategy from paper [0.91]
bash
$researchforge run --stall 3 --parallel 4
🚀 Running 7 experiments across 4 worktrees...
Your checkout: untouched.

The experiment dashboard

Here's what the ResearchForge Hub showed after all experiments completed:

ResearchForge Hub — rogii-wellbore · rmse (lower is better)
baseline: 15.2441
IDStatusHypothesisMetricΔ
exp-001PASShyp-005: Per-well normalization before feature engineering12.8903+2.35
exp-002FAILhyp-004: Self-supervised pre-training on unlabeled logs16.1122-0.87
exp-003PASShyp-006: Rolling window features at 3 scales (5, 20, 50)11.4421+3.80
exp-004REJECTEDhyp-007: Ensemble GBM + sequence, weighted by well type15.3301-0.09
exp-005PASShyp-002: Depth-normalized positional encoding9.2211+5.98
exp-006PASShyp-003: Multi-task (aux: gamma + resistivity prediction)7.8803+7.36
exp-007FAILhyp-001 v1: LSTM encoder — overfit on small per-well data18.4401-3.20
exp-008PASShyp-001 v2: GRU encoder + per-well norm (builds on exp-001)7.1104+8.13
exp-009WINNERhyp-001 v3 + hyp-003 + hyp-006: Combined sequence + multi-task + multi-scale6.1033+9.14

What the lineage showed

The rejection record was as valuable as the wins. exp-002 (self-supervised pre-training) failed because the unlabeled well logs I had access to came from different geological basins — the pre-trained representations transferred noise, not signal. That's a rejection with a reason. Without ResearchForge, I would have filed that away as "didn't work" and potentially tried variations of it again.

exp-004 (ensemble weighting by well type) was rejected — its improvement over baseline was within measurement noise (Δ = -0.09, below the 0.01 threshold). Not a failure, just not meaningfully better. The hub flagged it correctly.

exp-007 (LSTM) failed hard. But exp-008 combined the lesson from exp-007's failure (LSTM overfits on small per-well sequences → switch to GRU with shorter context) with exp-001's per-well normalization, and it worked. The lineage tracking made this "builds on failure" chain explicit.

The winning approach

exp-009 combined three validated ideas: GRU sequence encoder with depth-normalized positional encoding (hyp-001 v3), multi-task auxiliary prediction of gamma ray and resistivity (hyp-003), and multi-scale rolling window features at 5/20/50 steps (hyp-006).

The physical intuition: the GRU learns that the drill bit's position in the formation depends on where it came from as much as current sensor readings. Multi-scale features capture both sharp formation boundaries (5-step window) and gradual trends (50-step window). Predicting gamma and resistivity as auxiliary tasks forces the model to learn physics-consistent representations rather than overfitting to TVT alone.

Results

6.10
Final RMSE
was 15.24
60%
Improvement
RMSE reduction
~1,700
Rank
/ 6,125 teams
9
Experiments
3 rejected

Ship it

bash
$researchforge validate --experiment exp-009 --n 5
🔬 Validation run 1/5: rmse = 6.0891
🔬 Validation run 2/5: rmse = 6.1204
🔬 Validation run 3/5: rmse = 6.0977
🔬 Validation run 4/5: rmse = 6.1108
🔬 Validation run 5/5: rmse = 6.0985
✓ mean=6.1033 stdev=0.0114 n=5
✓ Stable. Approved for shipping.
$researchforge ship --branch feat/multi-task-gru-winner
✓ Branch created: feat/multi-task-gru-winner
✓ Report: .researchforge/reports/exp-009-final.json
✓ Only winning changes applied. Checkout clean.
💡
The key insight: The winning idea (sequence encoding with positional depth normalization) came from a paper I retrieved in the first search — one I'd never have found manually in a reasonable timeframe. ResearchForge didn't write the model. It made sure I was looking in the right place.