0 cumulative citations
View corpus contextAn RL-trained code retriever trims agentic exploration: CodeGrep, a 14B retrieval agent, cuts rounds by 15% and token consumption by 19% and raises resolve rates slightly (+1.2pp) when plugged into a frozen OpenHands coding agent, while weaker retrievers can actually harm performance.
CodeGrep: An RL-Trained Retrieval Agent for LLM Coding Agents
CodeGrep, a 14B RL-trained retrieval agent, when injected into a frozen OpenHands coding agent on SWE-Bench Verified, modestly raises resolve rate (+1.2 percentage points) while substantially reducing exploration rounds (−15%) and token use (−19%) on resolved issues.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
Modern LLM coding agents such as Claude Code and OpenHands share a common inefficiency: they spend much of their token budget finding the file to patch, rather than patching it. On SWE-Bench Verified, a 30B OpenHands agent averages 23 rounds and 631K tokens per resolved issue, with many calls spent on grep, glob, and view_file during repository exploration. We introduce CodeGrep, a 14B retrieval agent trained end-to-end with GRPO to issue multi-turn parallel grep, glob, and read tool calls and return candidate files to a frozen downstream coding agent. On all 500 SWE-Bench Verified instances, CodeGrep preserves resolve rate while substantially improving efficiency: 27.0% versus 25.8% for the no-retrieval baseline, with 15% fewer rounds and 19% fewer tokens on resolved instances. Across retrievers, downstream utility follows a precision threshold: BM25 with precision 0.375 degrades the agent, Jina with precision 0.445 is neutral, and CodeGrep with precision 0.677 crosses the threshold at which retrieval begins to reduce rollout cost. To enable this study, we mine supervision from 67K open-source agent trajectories using CATM and build a Git-worktree environment for multi-turn agent RL. In our setting, applying the efficiency signal at the advantage layer rather than the reward layer reduces KL drift and translates cleanly into downstream efficiency. We will release the model, training pipeline, RL environment, and evaluation harnesses.
Summary
Main Finding
CodeGrep — a 14B retrieval agent trained end-to-end with GRPO and injected into a frozen 30B OpenHands coding agent — meaningfully shortens the file-discovery phase of LLM coding agents and yields a clear efficiency win (−15% rounds, −19% tokens on resolved SWE-Bench Verified issues) while producing a small but reproducible resolve-rate lift (+1.2 percentage points, 25.8% → 27.0%). Retrieval quality maps to downstream utility via a precision threshold: low-precision retrievers can hurt the agent, mid-precision are neutral, and only sufficiently precise retrievers (CodeGrep v3: precision ≈0.677) buy net efficiency.
Key Points
-
System and model
- CodeGrep: 14B-parameter agent-style retriever built from Qwen3-14B-Instruct.
- Exposes three developer-like tools: grep, glob, read; can emit up to 8 parallel tool calls per turn; rollouts use up to 4 turns (3 exploration + 1 answer).
- Trained with GRPO (reinforcement learning) and LoRA on a single 8×B200 node (∼27 wall-clock hours per run; early-stop at step 897).
-
Data and supervision (CATM)
- CATM (Code Agent Trajectory Mining) mines relevance labels from 67,074 open-source OpenHands trajectories without human annotation.
- Pipeline: mine file-read events → judge-filter read reasoning with an LLM to label RELEVANT/NOT_RELEVANT → intensity-aware weighting by reasoning length.
- Training set after filtering: 31,977 effective samples. Target set per issue G(x) = gold-patch files ∪ CATM positives (weight thresholded).
-
RL / reward design (three iterations)
- v1: multiplicative efficiency penalty applied at the reward level (penalizes average tool calls per turn). Result: higher KL drift (∼0.31) and unstable training.
- v2: move efficiency scaling to the GRPO advantage layer (soft sqrt discount s(¯c) = sqrt(min(¯c/4,1))). Reduced KL drift (∼0.09) but introduced length-exploitation behavior.
- v3: drop the auxiliary line-range component (dead information for downstream) and keep advantage-layer efficiency scaling. v3 converged to higher reward, lower KL than v1, fewer tool-use turns, and better downstream performance.
-
Empirical performance (SWE-Bench Verified, 500 instances)
- Baseline (no retriever): resolve rate 25.8%, 23.0 rounds, 631K tokens per resolved issue.
- BM25 (precision 0.375): degrades downstream (resolve −0.6pp, tokens +21%).
- Jina-1.5B dense retriever (precision 0.445): roughly neutral (no resolve lift; modest token savings).
- CodeGrep v3 (precision 0.677): resolve 27.0% (+1.2pp), resolved rounds 19.6 (−15%), resolved tokens 514K (−19%).
- Across retrievers the observed regimes are: below-precision-threshold (hurts), near-threshold (neutral), above-threshold (buys efficiency).
- CodeGrep v3 produces high-quality retrievals (Fβ≥0.8) on 43.0% of instances, far above BM25/Jina (7.0%).
-
Engineering enablers
- Git-worktree lightweight sandbox replaced heavyweight Docker images; per-rollout setup reduced from minutes to milliseconds, enabling multi-turn RL training on a single node.
- Open release planned: model, CATM pipeline, RL environment, and evaluation harnesses.
Data & Methods
-
Dataset / Evaluation
- Training supervision from 67K+ OpenHands agent trajectories (CATM) producing 31,977 training cases after judge filtering and retention thresholds.
- Downstream end-to-end evaluation on SWE-Bench Verified (500 instances) using a frozen Qwen3-30B OpenHands agent as the code-writing module (100-round max, temperature 0).
- Retrieval intrinsic metrics: Fβ (β=0.5), file-level precision/recall, fraction with Fβ≥0.8. Downstream metrics: resolve rate, average rounds and tokens on resolved instances.
-
Model & Tools
- Retriever base: Qwen3-14B-Instruct with LoRA.
- Tool primitives: grep (regex), glob (path patterns), read (file contents). Parallel tool-call budget: up to 8 per turn; exploration limited to 3 turns.
-
RL training
- Algorithm: GRPO adapted to multi-turn parallel tool use.
- Reward design: primary objective is a precision-biased file-level Fβ (β=0.5). Efficiency introduced via scaling on reward (v1) or on advantage (v2/v3). v3 removes the line-range auxiliary term.
- Advantage-layer scaling used s(¯c) = sqrt(min(¯c/4,1)) to penalize excessive tool calls softly at the gradient stage, preserving stability and limiting KL drift.
-
Environment & infra
- Git-worktree sandbox: reuse bare repos and create lightweight worktrees per commit to enable fast, local grep/glob/read calls with path traversal protection and concurrency/timeouts.
- Training run: single 8×B200 with optimizer offload, vLLM colocated rollout; ∼27 hours per training run, early-stopped at step 897.
Implications for AI Economics
-
Cost-effectiveness of modular retrieval
- A relatively small (14B) specialized retriever can reduce downstream token consumption per resolved issue by ~19% (631K → 514K tokens), which directly lowers inference billing and latency when the downstream writer is a larger LLM (30B+). For workloads dominated by token costs, investing in a retriever module can be highly levered.
- The improvement in rounds (−15%) also reduces wall-clock latency and the number of downstream model invocations, further compounding cost savings.
-
Diminishing returns & a precision threshold
- Returns to investing in retrieval quality are non-linear: below a precision threshold, retrieval makes things worse; near the threshold, gains are neutral; above it, improvements mainly compress rollouts rather than expand the set of solvable issues. This implies a clear investment rule: bring retriever precision past the threshold first (to avoid negative returns), then expect gains mostly in reduced cost/latency rather than large increases in task coverage.
- For procurement / R&D prioritization, prioritizing methods that reliably cross the threshold (behavioral supervision like CATM + RL tuning) is more valuable than incremental gains on already-low-precision baselines.
-
Training stability matters for economic tradeoffs
- Reward design and where efficiency signals are injected materially affect policy drift (KL) and training stability. Advantage-layer scaling (v2/v3) kept KL lower and produced more reliable downstream gains. Lower KL drift reduces the risk of training catastrophic or unexpected behaviors, which lowers deployment risk and associated monitoring/rollback costs.
-
Cheap supervision is viable and scalable
- CATM mines labels from agent behavior without human annotation. This reduces labeling cost and enables large-scale training data for retrievers tailored to agent-style use. Economically, this lowers the marginal cost of building specialized retrievers compared to extensive human-labeled relevance datasets.
-
Operational considerations & risk
- Low-quality retrievers can harm downstream performance; thus, retriever deployment must be evaluated in end-to-end agent stacks, not only via intrinsic IR metrics. Deployment pipelines should include A/B tests with downstream agents and safety/rollback thresholds keyed to precision and token-efficiency metrics.
- The marginal benefit of further improving retrieval precision beyond the threshold accrues primarily as reduced compute/tokens per issue; decisions whether to invest further should weigh the incremental engineering/training cost against those operational savings.
-
Product strategy insight
- Rather than always scaling the main LLM writer, it can be economically preferable to invest in smaller specialized modules (retrievers, tool-use controllers) that cut expensive downstream inference. The CodeGrep result exemplifies modular optimization that yields outsized operational savings relative to model size.
Limitations to bear in mind - The resolve-rate gains are modest (+1.2pp). The biggest win is efficiency, not task coverage expansion. - Results are measured on SWE-Bench Verified with a specific downstream agent (OpenHands + Qwen3-30B). Generalization to other agents, repo types, or user workflows should be validated. - CATM relies on the quality of mined trajectories and the LLM judge; judge biases may affect label quality.
Release - The authors plan to open-source the CodeGrep model, CATM pipeline, RL environment, and evaluation harnesses — lowering reproducibility costs and enabling economic comparisons.
Assessment
Paper Typedescriptive
Evidence Strengthmedium — The paper reports controlled, reproducible experiments (500 held-out SWE-Bench Verified instances) with multiple baseline retrievers, internal retrieval metrics, and downstream injection into a frozen agent, plus ablations (v1→v3) that isolate reward-design effects. However, evidence is limited to one benchmark dataset and one downstream agent configuration, relies on automatically mined labels (LLM judge + heuristics) which may introduce noise/bias, and the downstream performance gains on resolve rate are small (+1.2pp), so external validity and robustness across other repos/agent architectures remain uncertain.
Methods Rigormedium — Strong engineering and experimental design: careful RL environment (git-worktree sandbox), trajectory-mining pipeline (CATM) with judge filtering and intensity weighting, multiple baselines, and systematic reward-design ablations; training details and diagnostics are provided. Limitations include reliance on an LLM judge for labels (potential systematic bias), evaluation on a single benchmark and a single frozen downstream model, early stopping of training runs, and limited human evaluation beyond internal audits.
SampleTraining data: 67,074 open-source OpenHands agent trajectories mined by CATM; after judge filtering and heuristics, 31,977 effective training samples. Model: CodeGrep (14B, Qwen3-14B-Instruct base with LoRA). Evaluation: SWE-Bench Verified (500 instances) with a frozen OpenHands downstream agent using Qwen3-30B-A3B-Instruct-2507; comparisons include BM25 and Jina-1.5B retrievers. Training run: single 8×B200 node, ~27 hours per run, early-stopped at step 897.
Themesproductivity human_ai_collab
IdentificationEmpirical ablation: train only the retrieval module (CodeGrep) with RL on mined relevance labels and inject its outputs into a frozen downstream OpenHands agent; compare downstream resolve rate and efficiency across configurations (baseline, BM25, Jina, CodeGrep v1/v2/v3) on SWE-Bench Verified to attribute changes to the retriever.
GeneralizabilityEvaluated on a single benchmark (SWE-Bench Verified) and one downstream agent, so results may not generalize to other repos, programming languages, or coding-agent architectures., Training labels come from mined agent trajectories judged by an LLM and heuristics (CATM), which can introduce label noise and systematic biases versus human annotation., The downstream agent is frozen (Qwen3-30B); interactions could differ if the agent were fine-tuned jointly or replaced by other LLMs or human developers., Model and training choices (14B parameter retriever, GRPO, worktree sandbox) may not scale identically across different compute budgets or production settings., Early stopping and single-node runs raise the possibility that longer training or different hyperparameters would change outcomes.
Claims (9)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| The baseline Qwen3-30B OpenHands agent resolves 25.8% of SWE-Bench Verified instances, averaging 23.0 rounds and 631K tokens per resolved issue. Organizational Efficiency | negative | Rounds and token usage required per resolved issue |
Reading fidelity
high
Study strength
high
|
n=500
23.0 rounds and 631K tokens per resolved issue
|
| Injecting CodeGrep v3 into the frozen OpenHands agent increases resolve rate from 25.8% to 27.0%, while reducing rounds on resolved instances by 15% and tokens by 19%. Organizational Efficiency | positive | Issue resolve rate, rounds per resolved issue, and tokens per resolved issue |
Reading fidelity
high
Study strength
high
|
n=500
+1.2pp resolve rate; −15% rounds; −19% tokens
|
| BM25 retrieval degrades downstream agent performance: resolve rate falls by 0.6 percentage points and resolved-token usage increases by 21% relative to the baseline. Organizational Efficiency | negative | Resolve rate and token usage per resolved issue |
Reading fidelity
high
Study strength
high
|
n=500
−0.6pp resolve rate; +21% resolved tokens
|
| Jina-1.5B retrieval is approximately neutral for resolve rate, leaving it at the 25.8% baseline, while reducing resolved tokens by 7%. Organizational Efficiency | null_result | Resolve rate and token usage per resolved issue |
Reading fidelity
high
Study strength
high
|
n=500
0pp resolve-rate change; −7% resolved tokens
|
| Across the evaluated retrievers, downstream utility follows a precision threshold: BM25 at precision 0.375 hurts performance, Jina at 0.445 is roughly neutral, and CodeGrep v3 at 0.677 provides an efficiency benefit. Organizational Efficiency | mixed | Downstream resolve rate, rounds, and token usage as a function of file-retrieval precision |
Reading fidelity
high
Study strength
medium
|
n=500
Precision 0.375, 0.445, and 0.677 across BM25, Jina, and CodeGrep v3
|
| CodeGrep v3 achieves higher retrieval quality than BM25 and Jina-1.5B, with mean Fβ of 0.576, precision of 0.677, and high-quality retrieval (Fβ ≥ 0.8) on 43.0% of evaluation instances. Output Quality | positive | File-retrieval Fβ score, precision, and proportion of high-quality retrievals |
Reading fidelity
high
Study strength
medium
|
Fβ mean 0.576; precision 0.677; Fβ ≥ 0.8 on 43.0% of instances
|
| Among the three CodeGrep reward-design iterations, v3 has the highest reported training reward, lowest stabilized tool-use turn count, and lower KL drift than v1. Training Effectiveness | positive | Training reward, KL policy drift, and average tool-use turns |
Reading fidelity
high
Study strength
medium
|
n=3
v3 reward 0.60–0.65 vs. 0.45–0.48 for v1/v2; v1 KL ∼0.31 vs. v3 ∼0.15; v3 turns ∼2.1
|
| Applying the efficiency signal at the advantage layer rather than the reward layer reduces reported GRPO policy drift, with KL of 0.09 versus 0.31. Training Effectiveness | positive | KL divergence from the reference policy during GRPO training |
Reading fidelity
high
Study strength
medium
|
n=3
KL 0.31 vs. 0.09
|
| The CATM pipeline mines supervision from 67,074 open-source OpenHands trajectories and produces 31,977 effective training samples after filtering. Training Effectiveness | positive | Number of trajectory records and effective training samples used for retriever training |
Reading fidelity
high
Study strength
medium
|
n=67074
31,977 effective training samples; 47.7% retention
|