0 cumulative citations
View corpus contextA 4B extractive compressor cuts coding-agent context to about a quarter of its original size while maintaining most problem-solving ability — Paritok-4B compresses to ~25.7% of tokens with ~86.5% single-shot solve retention and copies >96% of identifier-like tokens; self-hosting the small LoRA adapter is also cheaper than using a frontier model like gpt-5 as the compressor.
Paritok-4B: Intent-Conditioned Context Compression for Coding Agents
Paritok-4B is a 4B-parameter, intent-conditioned extractive compressor for coding-agent context that reduces context to ~25% of tokens while retaining roughly 86–89% of single-shot solve quality and preserving nearly all must-keep identifiers via copy-first output constraints.
Citation observations
Cumulative provider counts captured on specific dates; providers are never combined.
Coding agents re-send large file reads and tool outputs to a frontier LLM every turn, and this context dominates their token bill. General-purpose prompt compressors are trained on prose and suit code poorly: they paraphrase identifiers and drop the exact spans an agent needs to edit. We present Paritok-4B, a 4B LoRA compressor for coding-agent trajectories built on two commitments. It is extractive: it selects spans rather than rewriting them, and 96.0% of the identifiers, paths, and numbers it emits already appear in its input, holding at 96.2% on held-out SWE-bench Lite output. It is intent-conditioned: told the agent's current task, it acts chiefly inside a retained segment, selecting which lines survive (retained lines are +0.067 more intent-relevant than removed ones, paired 95% CI [+0.056, +0.078]) rather than changing how much is retained. We distil a gpt-4.1-mini teacher over 67,074 real OpenHands trajectories into 40,606 validated examples and fine-tune Qwen3-4B. On all 300 SWE-bench Lite instances, Paritok-4B compresses agent context to 25.7% of its size, 2.0x harder than a gpt-4.1-mini compressor (50.2%) and 2.4x harder than gpt-5 (61.9%), while retaining 86.5% of uncompressed single-shot solve quality. Fed the cat -n line-numbered input real agents produce, it compresses slightly less (27.8%) and retains more (89.3%); there the paired test is informative, with 30 instances solved only uncompressed and 17 only compressed, an exact McNemar p=0.079, so at this sample size compressing context to roughly a quarter of its size does not significantly reduce the solve rate. The model is a 264 MB adapter that self-hosts on one 24 GB GPU with no per-token compressor fee, which at list prices decides the economics: gpt-5 as a compressor is net-negative, costing more than the downstream tokens it saves. Weights, data, and evaluation scripts are open (Apache 2.0).
Summary
Main Finding
Paritok-4B is a 4B-parameter, intent-conditioned, extractive LoRA compressor tailored to coding-agent trajectories. By compressing agent context per typed segment (extracting spans rather than paraphrasing, and conditioning on the agent’s current intent), it reduces context sent to frontier LLMs to roughly a quarter of its original token size while preserving most task performance and preserving exact spans needed for reliable code edits. The model is small (264 MB LoRA adapter), self-hostable on a single 24 GB GPU, and cost-effective compared with using a frontier LLM as a compressor.
Key Points
- Design commitments
- Extractive: selects/copies spans rather than rewriting. Measured copy rates: 96.0% of emitted identifier-like tokens are present in input on the distilled corpus; 96.2% on held-out SWE-bench Lite outputs. Line-level: 82.3% emitted lines are byte-identical to input; 3.2% are closed-vocab markers; 14.5% novel (many by-design abstractive cases).
- Intent-conditioned: compressor receives the agent’s current task/query and prioritizes retaining entities named by that intent (retained lines are more intent-relevant by +0.067, 95% CI [+0.056, +0.078]).
- Per-segment operation: gateway splits requests into typed segments (kinds like file_read, tool_result, log_output, etc.) and compresses each independently. This enables short model calls (median prompt ≈4K tokens), parallelism, incremental recompression, and per-segment recoverability.
- Performance (selected metrics)
- Out-of-distribution (OOD) holdout: 100% well-formed outputs per segment, compresses to 23.7% of input tokens at the evaluated budget; student’s must-keep identifier retention comparable to teacher on matched-budget segments (example paired numbers reported: 0.385 vs. 0.287; directionally favorable but small-sample non-significant at n=39).
- End-to-end SWE-bench Lite (300 instances): compresses full agent context to 25.7% of original size, while retaining 86.5% of uncompressed single-shot solve quality.
- In-distribution (cat -n framing) regime: compresses to 27.8% and retains 89.3% solve quality. Paired solve differences across 300 instances were not statistically significant (McNemar exact p = 0.079).
- Comparative compression difficulty: Paritok-4B’s compression is ~2.0× more aggressive than a gpt-4.1-mini compressor (gpt-4.1-mini kept 50.2% of tokens) and ~2.4× more aggressive than gpt-5 (61.9%).
- Practical/economic points
- The compressor is a 264 MB LoRA adapter on a Qwen3-4B backbone (chosen over 3B and 7B alternatives in matched protocol).
- Self-hosting avoids per-token compressor fees; authors show using an expensive frontier model (gpt-5) as a compressor can be net-negative—its cost can exceed the downstream token savings.
- Weights, data pipeline, and evaluation scripts (including extractiveness audit) are open-sourced (Apache 2.0).
Data & Methods
- Data pipeline (reproducible five-stage funnel; every script re-runnable)
- Source: 67,074 OpenHands agent trajectories (SWE-rebench and SWE-Gym). SWE-bench Lite held out for evaluation.
- Segmentation: trajectories split at assistant decision points into sequences of typed segments (system prompt, user task, file reads with cat -n framing, tool outputs, edits, reasoning blocks).
- Filtering: keep turns with a real action and at least one compressible segment (≥1,000 tokens). Over-long segments split; requests >32K middle-truncated with a retention policy.
- Labeling: automatic L0–L3 importance labels, kind tags, must-keep span extraction (paths, identifiers, error classes, line numbers, code keywords), stale-file detector.
- Pooling: capped at 80K train / 4K val turns; stratified sampling by length, action type, etc.
- Distillation & validation: extracted segments (45K candidate segments; 40,606 validated) distilled via a gpt-4.1-mini teacher (T=0) with two teacher prompts (file_read vs other kinds). Teacher outputs were validated against budgets and other quality checks; 90.2% of candidates passed validation.
- Model & training
- Student: Qwen3-4B backbone fine-tuned with LoRA (resulting adapter 264 MB).
- Selected after matched-protocol comparisons with 3B and 7B code-pretrained alternatives.
- Output format: each compressed segment enclosed in [SEG id=...] markers so gateway can map back to originals; closed vocabulary of structural markers is specified in the system prompt (Table of markers provided).
- Extractiveness rules: certain classes (code lines, identifiers, paths, imports, error-message text, shell commands, old/new payloads) must be copied verbatim; bounded rewriting allowed only in prescribed contexts (e.g., truncated string literals in logs, assistant_thinking condensed to ≤200 characters).
- Empirical audits & failure analyses
- Extractiveness audit run on both training-distilled corpus and held-out SWE-bench Lite; notable residual invention analyzed (teacher sometimes synthesizes fully-qualified names or restructures expressions).
- Level labels (L0–L3) largely collapsed in practice into two bands: protected/recent (~0.40 median realized) vs stale (~0.20). Table: realized median ratios L0=0.379, L1=0.408, L2=0.198, L3=0.133; only L1/L2 boundary robust. Authors report this design claim failed to hold and provide diagnostics.
- Teacher prompt was tuned iteratively (human hill-climb against gold compressions) and gpt-4.1-mini chosen for cost/quality trade-off.
Implications for AI Economics
- Compressor-as-infrastructure reduces recurrent token bills to expensive frontier LLMs:
- Small self-hosted compressors (LoRA adapters on 4B backbones) can drastically cut the tokens sent to a paid frontier model without per-token compressor costs, producing strong ROI when the expensive LLM is billed per token.
- The paper provides an explicit instance where using an expensive frontier LLM (gpt-5) as the compressor is net-negative—compression cost exceeds the downstream token savings—highlighting that cheaper, self-hosted compressors can be economically dominant.
- Task-specific, extractive compressors increase operational reliability/value for coding agents:
- Extractive guarantees (copy-first) reduce edit failures caused by paraphrase or renaming, increasing the operational value of compression for editing agents and lowering the risk/cost of failed downstream edits.
- Conditioning on live intent leverages free, high-value signal available in agent systems; this raises the effective value per compressed token compared with task-agnostic compressors.
- Deployment trade-offs
- Per-segment compression enables parallelism, incremental recomputation, and recoverability—operational features that reduce latency and troubleshooting cost, and that improve economic feasibility for real-time agent pipelines.
- The failure of the L0–L3 granularity shows a practical limit: supervision and teacher targets can collapse intended policy granularity, so simpler level bands may be sufficient in practice—this affects how much complexity operators should build into labeling schemes.
- Open release lowers friction and cost for adoption
- Apache 2.0 release of weights and pipelines reduces integration costs and supports reproducible economic analysis by practitioners.
- Broader economic considerations
- Coordinating a small compressor with a frontier model changes the marginal economics of using large LLMs in agent loops: compression can turn otherwise-prohibitive token costs into feasible workflows, but only if the compressor itself is cheap enough (compute + hosting) and preserves task quality.
- Future cost analyses should include end-to-end agent costs (compute, latency, engineering complexity) rather than only compressor vs downstream token trade-offs; the authors explicitly note their harness measures comprehension under compression rather than full agent operational cost.
If you want, I can: - Extract the key numeric tables (compression ratios, extractiveness by kind, level realizations) into a concise table for quick reference. - Produce a short checklist for deploying Paritok-4B in an existing coding-agent pipeline (gateway integration, safety/recovery checks, monitoring signals).
Assessment
Paper Typedescriptive
Evidence Strengthmedium — The paper provides a reproducible pipeline, teacher-distilled supervision, held-out OOD evaluation (300 SWE-bench Lite instances), extractiveness audits on large token counts, and statistical reporting (CIs, McNemar test). However, results depend on a distilled teacher, some held-out repositories may overlap with training corpora, the end-to-end economic claims are not measured in deployment, and several comparisons (e.g., a direct head-to-head versus competing compressors under identical inputs) are omitted, limiting external validity.
Methods Rigorhigh — The authors document a scripted, multi-stage, re-runnable data funnel, stratified train/val splits at trajectory level, explicit validation gates for teacher outputs, OOD holdouts, quantitative audits of extractiveness at line/token granularity, and baseline comparisons across backbone sizes and frontier compressors; they also report CIs and hypothesis tests. Weaknesses include teacher-dependence of targets, some admitted design failures (level labels collapsed), incomplete head-to-head baselines under identical protocol, and limited sample sizes for some statistical tests.
SampleTraining signal distilled from a gpt-4.1-mini teacher over 67,074 OpenHands agent trajectories, yielding a 45K-segment distillation pool and 40,606 validated segment-level examples; per-segment kinds include file_read, tool_result, log_output, etc. Intrinsic audits run over the full distilled corpus (236K emitted lines) and token-level extractiveness; main end-to-end evaluation uses 300 held-out SWE-bench Lite instances (64,843 emitted lines, 212,506 identifier-like tokens). Baselines include gpt-4.1-mini and gpt-5 compressors; the student model is a Qwen3-4B backbone fine-tuned with LoRA (264 MB adapter).
Themesproductivity human_ai_collab
GeneralizabilityEvaluation limited to SWE-bench Lite and OpenHands-derived trajectories; held-out instances are unseen instances but some repositories may appear across corpora (not fully unseen-repo generalization)., Distillation depends on a gpt-4.1-mini teacher and its prompt; student behavior may inherit teacher biases and failure modes., Design assumes coding-agent inputs with cat -n framing and typed segments—approach may not transfer to non-code or non-agent contexts (prose, other tool outputs)., Level-labeling scheme collapsed empirically, indicating some intended controls do not generalize across distributions., End-to-end economic conclusions (net cost savings vs frontier compressor fees) are scenario-specific and not measured in deployed production settings with real traffic dynamics.
Claims (12)
| Claim | Direction | Outcome | Confidence & Evidence | Details |
|---|---|---|---|---|
| Paritok-4B is substantially extractive: 96.0% of emitted identifier-like tokens, dotted paths, and numbers in the distilled corpus already appear in the input; this rises to 98.3% when assistant_thinking is excluded. Output Quality | positive | Fraction of emitted identifier-like tokens copied from the input |
Reading fidelity
high
Study strength
medium
|
n=236152
96.0% token copy; 98.3% excluding assistant_thinking
|
| The extractive behavior generalizes to held-out SWE-bench Lite outputs: 96.2% of emitted identifiers, paths, and numbers already appear in the input. Output Quality | positive | Token-copy rate on held-out coding-agent outputs |
Reading fidelity
high
Study strength
medium
|
n=300
96.2% token copy over 212,506 emitted identifier-like tokens
|
| Intent conditioning primarily changes which lines are retained rather than the overall amount retained: retained lines were 0.067 more intent-relevant than removed lines. Task Allocation | positive | Intent relevance of retained versus removed lines |
Reading fidelity
high
Study strength
medium
|
+0.067 intent-relevance difference, paired 95% CI [+0.056, +0.078]
|
| The training pipeline used 67,074 real OpenHands trajectories and produced 40,606 teacher-validated compression examples from a 45,000-segment distillation pool. Training Effectiveness | positive | Scale of the training and distillation dataset |
Reading fidelity
high
Study strength
medium
|
n=67074
40,606 validated examples from 45,000 candidates
|
| On an out-of-distribution holdout, the released checkpoint produced well-formed output on every segment and compressed inputs to 23.7% of their original token count. Organizational Efficiency | positive | Output format validity and compression ratio |
Reading fidelity
high
Study strength
medium
|
100% well-formed output; 23.7% of input tokens
|
| At a comparable compression budget on an out-of-distribution holdout, Paritok-4B did not show lower must-keep identifier retention than its teacher. Output Quality | null_result | Must-keep identifier retention |
Reading fidelity
high
Study strength
low
|
n=39
0.385 student retention vs. 0.287 teacher retention; paired difference not significant
|
| Across all 300 SWE-bench Lite instances, Paritok-4B compressed agent context to 25.7% of its original size while retaining 86.5% of uncompressed single-shot solve quality. Output Quality | mixed | Context compression ratio and single-shot software issue solve quality |
Reading fidelity
high
Study strength
medium
|
n=300
25.7% context size; 86.5% retained solve quality
|
| Paritok-4B achieves substantially more aggressive compression than the compared GPT compressors while maintaining comparable single-shot solve quality: 25.7% of input tokens versus 50.2% for gpt-4.1-mini and 61.9% for gpt-5. Organizational Efficiency | positive | Relative context-token usage at comparable solve quality |
Reading fidelity
high
Study strength
medium
|
n=300
25.7% versus 50.2% and 61.9% of input tokens; approximately 2.0× and 2.4× more aggressive compression
|
| In the in-distribution cat -n regime, Paritok-4B compressed context to 27.8% of its original size and retained 89.3% of uncompressed solve quality. Output Quality | mixed | Context compression ratio and retained solve quality under line-numbered agent input |
Reading fidelity
high
Study strength
medium
|
n=300
27.8% context size; 89.3% retained solve quality
|
| The paired comparison in the cat -n regime did not establish a statistically significant reduction in solve rate from compression. Output Quality | null_result | Paired compressed versus uncompressed solve rate |
Reading fidelity
high
Study strength
medium
|
n=300
Exact McNemar p=0.079; 30 uncompressed-only versus 17 compressed-only solves
|
| The intended four-level compression-budget design did not produce four distinct realized compression bands; instead, the distilled targets formed roughly two bands separating protected/recent from stale context. Other | null_result | Separation of realized compression ratios by intended importance level |
Reading fidelity
high
Study strength
medium
|
n=28248
P(L0 < L1)=0.534; P(L1 < L2)=0.280; P(L2 < L3)=0.456
|
| At the line level, 82.3% of emitted lines in the distilled corpus were byte-identical to an input span, 3.2% were closed-vocabulary markers, and 14.5% were newly generated. Output Quality | positive | Line-level verbatim copying and generation composition |
Reading fidelity
high
Study strength
medium
|
n=236152
82.3% verbatim; 3.2% markers; 14.5% novel
|