A small independent team used a data-centric post-training pipeline to lift open-weight models’ cyber skills — raising CyberGym verified success by about 24% on average and placing a checkpoint in the top 10 overall while topping peers at comparable parameter scales; their toolkit combines teacher-extraction, API-bypass, model-merge reversal and human-expert trajectory internalization, but several methods depend on provider-specific and ethically sensitive techniques.

Main Finding

A seven-person independent team (Vera Praxis Lab) used a data-centric, environment-grounded post-training pipeline and five complementary techniques to turn open-weight Qwen checkpoints into leading agentic cyber models. They produced three Feyospace checkpoints (s0, s1, s2) trained on 164,269 execution-verified, audited trajectories spanning coding, vulnerability/exploit, CTF, kernel-history, firmware, and device-backed environments. Across the three checkpoints, supervised fine-tuning raised verified CyberGym performance by an average of 23.76% and pooled CTF success by 10.49%. As of 2026-09-01, Feyospace-s1 achieved a 63.24% verified success rate (10th on CyberGym overall) and all three checkpoints rank first among models at comparable parameter scales.

Key Points

• Five complementary supervision/capability techniques:
  • Choulea: recovers and analyzes hidden model reasoning "signatures" (atomic operations and compositions) to study reasoning fidelity and process supervision.
  • SkyReal: dramatically reduces teacher-sampling costs via low-cost account leverage and service-market arbitrage.
  • Hongzwang: elicits useful behaviour from closed/proprietary teachers under API/context/length restrictions using mutation strategies, structured retries, and task-specific skills.
  • PSBreakup: white-box model-merge reversal to restore latent or weakened target-domain capabilities in open-weight checkpoints before SFT.
  • Kreator: converts critical human expert interventions into coherent, teacher-native multi-turn trajectories suitable for agentic SFT.
• Environment-grounded data engine:
  • Repository-level coding: processed 27,502 verified trajectories from 3,751 repos (scanned ~2.2M PRs).
  • Security environments: 69,854 in-the-wild CVE reconstructions (Category A), 9,312 author-maintained CTFs (B), 12,993 kernel-history mined environments (C), and 1,601 reconstructed full-exploit cases (D).
  • Hardware/IoT: 1,003 firmware re-hosts and 374 physical device-backed environments.
  • Candidate trajectories only admitted after execution verification and a four-layer evidence audit (deterministic screen, leak/reward-hack checks, authenticity, behavior/log analysis). Audit outcomes: ~85% accept, 9% review, 6% reject.
• Data and training mixture:
  • Final audited corpus: 164,269 trajectories (28,177 coding; 136,092 security/hardware).
  • Teacher rollouts: typically 1–3 samples per environment; each trajectory execution-verified (gates for completion, PoC/flag, exploit evidence).
  • Post-trained checkpoints: Qwen3.6-35B-A3B, Qwen3.8-27B, Qwen3.5-122B-A10B → Feyospace-s0/s1/s2 using SFT-only long-context supervised fine-tuning and distributed training with long-context packing.
• Empirical outcomes:
  • Average improvement over starting checkpoints: +23.76% CyberGym verified success rate; +10.49% pooled CTF success.
  • Feyospace-s1: 63.24% verified success (2026-09-01), 10th on CyberGym; all three Feyospace checkpoints are top-ranked among same-scale models.
• Signature Hack (Choulea findings):
  • Authors describe methods to recover encrypted or hidden reasoning traces returned by some proprietary APIs; characterize atomic reasoning operations and "cognitive dialects".
  • Reported recoverability: pre-defense weighted aggregate ~92.8%; after provider defenses (post-Aug 21, 2026) Generation 5 recovery ~67% for traces <4096 tokens and ~39% for longer traces.
  • Recovered traces were used for analysis and method development, not for SFT training (due to resource, contamination, and data-quality concerns).
• Ethics/positioning:
  • Authors state an explicit motivation to broaden access and critique provider secrecy; however, some techniques (e.g., Hongzwang, Signature Hack, SkyReal) involve eliciting or leveraging behaviours that may bypass provider restrictions — raising legal/ethical concerns.
  • The team intends to release the audited training trajectories to support reproducibility.

Data & Methods

• Data construction pipeline:
  • Multi-source environment creation (repos, CVEs, CTFs, kernel histories, full exploit reconstructions, firmware re-hosts, physical devices).
  • Execution harness: automated agent harness executes candidate trajectories in resettable sandboxes or on device emulators; verifies completion and proof-of-concept artifacts (flags, exploits), and collects logs/evidence.
  • Four-layer evidence audit (L1 deterministic checks; L2 leak & reward-hack detection; L3 authenticity; L4 behavioral/log inspection).
  • Retention policy: only execution-verified and audited trajectories enter the training mixture.
• Supervision acquisition and filtering:
  • Teacher rollouts from open and closed models, plus human expert interventions converted via Kreator.
  • Cost-reduction and access methods: SkyReal (cheap account / token arbitrage), Hongzwang (mutation + retry workflows to navigate API constraints and refusal behaviors).
  • PSBreakup: model-merge reversal techniques to restore capabilities in open models prior to SFT.
  • Choulea: signature extraction pipeline to analyze internal reasoning traces, identify atomic operations and higher-level compositions; used for measurement and to shape data curation (not used directly as labels).
• Training:
  • SFT-only objective on long-context supervised trajectories; long-context packing and distributed training across checkpoints.
  • Models post-trained: three Qwen family checkpoints of varying sizes; evaluation primarily on CyberGym and pooled CTF suites.
• Evaluation:
  • Execution-verified success rate measured via CyberGym suite; pooled CTF evaluation also reported.
  • Leaderboard placement (CyberGym) and within-parameter-scale comparisons used to benchmark progress.
• Reproducibility:
  • Authors plan to release training trajectories; technical specifics of distributed training, hyperparameters, and exact compute costs summarized in appendices (paper claims an end-to-end account of engineering choices).

Implications for AI Economics

• Entry-cost dynamics and diffusion of frontier capabilities:
  • Demonstrates that small, well-engineered teams can substantially upgrade open-weight checkpoints into frontier-capable domain specialists through data-centric post-training rather than full-from-scratch pretraining. This lowers some barriers to capability diffusion and changes capital requirements (more emphasis on high-quality environments and supervision engineering than raw compute).
• Data vs compute economics:
  • Reinforces that the marginal cost of high-quality, execution-verified training data (environments, teacher rollouts, audits) can dominate post-training budgets. Techniques like SkyReal (account/token arbitrage) materially affect per-sample costs and thus the feasible scale of supervision.
• Market for environment and verification services:
  • Creates value for providers of resettable, high-fidelity emulated environments, firmware/device hosting, and automated evidence-audit tooling. Commercial markets can emerge for certified, verifiable environment datasets and auditing-as-a-service.
• Incentives and strategic interactions with closed providers:
  • Methods that elicit or recover hidden reasoning (Signature Hack) or bypass API constraints (Hongzwang) create negative externalities and an arms race between model/service providers and adversarial elicitations. Providers may respond with tighter gating, reducing transparency and increasing access costs; conversely, demand for open, auditable models may rise.
• Labor and market impacts in cybersecurity:
  • Agentic cyber models that can reliably solve CTFs, exploit development, and firmware rehost tasks could substitute routine or mid-level security labor, shifting demand toward higher-level red-team design, audit, and oversight. They also create new markets (automated vulnerability triage, exploit synthesis as a service) with significant commercial and societal risk/value.
• Regulatory and governance considerations with economic effects:
  • Techniques discussed raise legal, ethical, and safety questions (e.g., exploiting provider outputs, bypassing access controls, creating agentic exploit-capable models). Regulation or industry norms limiting such techniques would change cost structures and the competitive landscape—raising compliance costs but potentially slowing harmful capability diffusion.
• Investment and public-good arguments:
  • The paper’s approach underscores the social value of publicly available, audited datasets and tooling for safe capability research. Funding public-data efforts and independent audit infrastructure could be a cost-effective policy lever to reduce concentrated control over dangerous capabilities and to align market incentives with safety and transparency.
• Caveats affecting economic interpretation:
  • Some techniques rely on activity that may be contested or legally risky; the long-term cost-effectiveness and replicability depend on provider countermeasures (already observed) and on whether the audited dataset the authors release is sufficiently general and high-quality.
  • Measured performance gains are specific to cyber/security benchmarks (CyberGym/CTF) and may not generalize to other domains without similar environment investments.

If you want, I can: (a) produce a one-page infographic-ready bullet list of the costs/resources the team reports (environment creation, audit effort, teacher sampling reductions), or (b) extract the concrete figures and timeline for Signature Hack generations and recoverability to support a policy-risk brief. Which would you prefer?