Cheap CPU classifiers can handle most LLM jailbreak screening: TF‑IDF SVMs/GBTs match transformer accuracy on in-distribution traffic at roughly one‑fifth the cost, but confidently miss out‑of‑distribution attacks; a three‑stage GuardChain (Regex → CPU → GPU) routes only the hard cases to expensive GPU models, resolving 80% of prompts without GPU inference.

Main Finding

CPU-classifiers (TF‑IDF + SVM/LightGBM/RandomForest) can safely resolve the majority of LLM jailbreak-detection traffic at far lower cost than running a GPU guard on every prompt, but they exhibit a distinctive failure mode on out-of-distribution attacks (confident miscalibration). Combining a cheap CPU stage with a compact GPU stage (a three-stage pipeline called GuardChain: Regex → CPU → GPU) recovers robustness while reducing GPU usage and deployment cost substantially (up to ~5×).

Key Points

• Evaluation scope
  • Five CPU classifier families (SVM variants, LightGBM, Random Forest) vs. GPU baselines (Mamba-130M SSM; transformer baselines DeBERTa-v3 and Gemma-2B with LoRA).
  • Nine jailbreak sources merged into a 43,523-sample corpus (≈55% malicious). Three regimes:
    • D1: in-distribution (standard random split);
    • D2: out-of-distribution (source-held-out test);
    • D3: adversarially obfuscated (leetspeak, character perturbation, invisible Unicode injection).
• Main accuracy/cost findings
  • On D1 (in-distribution): best CPU classifier (LightGBM) matches best transformer GPU model to within ~1 percentage point F1 while costing roughly one-fifth as much to serve.
  • On D2 (OOD): CPU classifiers collapse (F1 < 0.43). Failure mode is not just low confidence but "confident miscalibration" — many OOD malicious examples receive high-confidence benign predictions and therefore bypass escalation (72% of OOD attacks in their eval).
  • On D3 (obfuscated): CPU classifiers (e.g., LightGBM) outperform transformer GPU models by >26 percentage points F1. Mamba-130M (SSM) handles all regimes well (e.g., F1 ≈ 0.960 on D1 and 0.927 on D3).
  • Error complementarity: only ~23% overlap in errors between SVMs and Mamba on D1, motivating cascaded use.
• GuardChain pipeline
  • Three stages: Regex filter (very cheap, trivial pattern matches) → CPU classifier (TF‑IDF + LightGBM/SVM) → GPU model (Mamba-130M).
  • CPU stage alone resolves 80–95% of in-distribution and obfuscated traffic at near-peak accuracy.
  • Routing ~20% of inputs (including OOD-like inputs) to the GPU stage raises pipeline F1 on D2 from 0.435 to 0.573.
  • End-to-end cost improvements: up to ~5× reduction vs. running the GPU guard on every prompt. Per-request cost examples in their environment: CPU stage ≈ $0.17–2.77 per 1M requests; Mamba GPU ≈ $6.80 per 1M requests. (Transformer LoRA models were much more expensive: DeBERTa-v3 LoRA ≈ 45× SVM; Gemma-2B LoRA ≈ 80× SVM by their accounting.)
• Calibration & operational notes
  • SVMs used isotonic calibration; LightGBM thresholds were F1-optimized on validation splits.
  • Confident miscalibration from TF‑IDF features is an architectural issue (vocabulary missing from training gets near-zero TF-IDF weight), not solvable solely by threshold tuning.
  • Mamba (selective SSM) offers a middle ground: compact, linear-time inference, robust to OOD and obfuscation.

Data & Methods

• Corpus and splits
  • Aggregated nine public jailbreak sources (after deduplication: 43,523 samples). D1: 70/15/15 mixed split. D2: train on 4 sources, test on 5 held-out sources (designed to be qualitatively different attack templates). D3: same D1 examples transformed with stochastic leetspeak/character perturbation/unicode injections.
• CPU feature set and classifiers
  • TF‑IDF features over word n-grams, character n-grams, or both (feature vocab sizes ~5k–20k), augmented with eight engineered safety features (char_length, word_count, avg_word_len, punct_ratio, upper_ratio, digit_ratio, special_ratio, repetition).
  • Models: linear SVM (with isotonic calibration), LightGBM (class-imbalance correction, F1-optimized threshold selection), Random Forest (300 trees).
• GPU models
  • Mamba-130M: selective state-space model (SSM), 129M parameters, linear-time token processing, two-phase curriculum fine-tuning (general safety then jailbreak specialist).
  • Transformer baselines: DeBERTa‑v3 and Gemma‑2B fine-tuned with LoRA.
• Regimes & robustness tests
  • D1 (in-distribution), D2 (OOD by source), D3 (obfuscated via Tleet ◦ Tperturb ◦ Tunicode).
• Metrics & operational measures
  • Primary metric: F1 for the malicious class. Additional analyses on confidence distributions, calibration, per-stage latency, and estimated per-request deployment cost.
  • Inference-only evaluation; training procedures were outside the paper’s evaluation scope.

Implications for AI Economics

• Lower marginal serving costs for safety
  • Replacing a per-request GPU guard with a CPU-first cascade can reduce the need for GPU inference massively — authors report up to ~5× cost reduction in their setting, with CPU stages resolving ~80% of traffic.
  • Practical per-million-request cost examples (their environment): CPU stage ~$0.17–$2.77 / 1M, Mamba GPU ~$6.80 / 1M; transformer LoRA guards were tens of× more expensive.
• Capital and operational implications
  • Reduced GPU capacity needs: fewer GPU instances → lower capital/spot-instance spend, simpler capacity planning, reduced provisioning for peak loads if cascading and queuing are used.
  • Latency benefits: CPU-first stages have lower per-request latency in many environments (regex <0.2 ms; CPU 3–50 ms; GPU example 24.3 ms), improving user-facing responsiveness for most queries.
• Risk trade-offs and hidden costs
  • Confident miscalibration on OOD inputs is an economic and security externality: cheaper CPU-only pipelines can create silent failure modes where malicious prompts bypass detection with high confidence. That risk imposes potential costs (abuse, compliance fines, reputational damage) and motivates maintaining a GPU-backed escalation path.
  • Ongoing monitoring and retraining costs: to keep CPU stages effective, you must invest in continual dataset curation, OOD detection, and monitoring for confidence-drifts. These operational costs offset some raw serving-cost savings.
• Recommended economic strategy
  • Use a staged/cascaded deployment: cheap deterministic filters (regex) → CPU classifiers for the bulk of traffic → compact GPU (e.g., Mamba-130M) for low-confidence inputs and suspected OOD/adversarial traffic.
  • Provision GPU capacity sized for the expected escalation fraction (e.g., ~20% in their experiments) rather than for full traffic, yielding substantial savings while retaining robustness.
  • Track calibration and “confident benign” rates on held-out / newly observed sources; prioritize investment in a GPU escalation stage if confident miscalibration rates rise.
• Caveats for practitioners
  • Reported numbers depend on dataset composition, threat model, and deployment environment (their Appendix notes true CPU-only latency may be slower in some environments). Realized savings will vary; run a small-scale A/B and monitoring pipeline before full switch-over.
  • The CPU stage design (feature engineering, TF‑IDF vocab coverage, engineered signals) matters; naive lexical filters are brittle to obfuscation and OOD shifts.

Summary recommendation: For many production settings, it is economically optimal to avoid running a GPU guard on every request — instead deploy a conservative multi-stage pipeline that uses cheap CPU classifiers for the bulk of traffic and routes uncertain or unusual queries to a compact GPU model. This captures most cost savings while preserving robustness to OOD and adversarial inputs.