Paritok-official/paritok-4b-v1

A non-destructive compression gateway for coding agents — cut your input-token bill without changing your agent.

Paritok sits between your agent and the LLM as a drop-in proxy. On every request it strips the tool-schema bloat, compresses tool results and file reads, and summarizes stale history — then forwards upstream, billed on the compressed tokens. Nothing is ever permanently discarded: the agent pulls back any exact original on demand. Works with Claude Code, Cursor, Codex, OpenHands, and any agent that honors BASE_URL — you don't change a line of your agent.

Powered by the first open-source 4B compression model trained specifically for coding agents (45K real agent trajectories). Cut input token bills from ~25% on turn one to past 85% in long, context-saturated sessions, and fit ~3× more turns in the same context window.

What it does · The three levers · Compounding savings · Cost · Pricing · Quick Start · The engine (model) · Team

• 2026-08-17   v1.3.8 — a visual /stats dashboard (live token savings + per-request original→compressed before/after, in place of the raw JSON) and the new Paritok VS Code extension.
• 2026-07-31   v1.3.0 — stability release: edit-recovery, read_original API rename (from expand_context).
• 2026-07-19   v1.2.0 ships the embedding-based tool filter — the biggest single-turn lever (~29K → ~8K on a typical Claude Code turn), unlocking prompt-cache-friendly tool selection with gateway_search_tools recall.
• 2026-07-15   Paritok gateway v1.0.0 open-sourced — the proxy/middleware that turns the 4B model into a drop-in Claude Code / Cursor / Codex compression layer.
• 2026-07-14   Paritok-4B-v1 released on Hugging Face Hub with full SWE-bench Lite end-to-end evaluation.
• 2026-06-25   Finished training. 45K teacher-distilled samples on the Qwen3-4B backbone.

Paritok runs as a middle layer between your agent and the LLM API — your agent points at Paritok instead of Anthropic/OpenAI, and everything else stays the same.

Your Agent (Claude Code / Cursor / Codex)
  → builds request (tool schemas + history + tool results / file reads)
     ★ Paritok gateway rewrites the request here ★
  → forwarded to Anthropic / OpenAI  (billed on the compressed tokens)
  ← response flows back unchanged; compressed refs expand on demand

The token bill for a coding agent is dominated by input you re-send every turn: dozens of tool schemas, an ever-growing message history, and big file-read / tool-output blocks. Paritok attacks all three — and because it's a non-destructive gateway, anything it compresses or filters is still recoverable on demand. It's lossy on the wire, fully recoverable when it counts.

Paritok saves tokens through three independent mechanisms. They stack, and they hit different parts of the bill:

Coding agents expose dozens of tools — often 70+ once you add MCP servers — in full JSON schema on every request. Most are irrelevant to the task at hand. Paritok filters them semantically (tool_discovery.strategy: embedding), keeping only the handful relevant to the user's intent in full schema and stubbing the rest.

• This is the largest single-turn lever. On a typical Claude Code turn the tool block alone is ~29K tokens; filtered it drops to ~8K — a saving no amount of file compression matches on a single turn.
• Prompt-cache friendly. The selection is frozen per conversation, so the tools[] block stays byte-stable turn-to-turn and never invalidates the LLM's KV cache.
• Never destructive. Anything filtered is recoverable — the model calls gateway_search_tools and gets the full schema back.
• Runs a small open embedding model (BAAI/bge-small-en-v1.5, MIT, ~130MB) entirely locally on CPU — no API, no per-token fee.
• An agent's core execution tool (shell / exec / apply_patch) is never stubbed, so agents like Codex that expose only a handful of tools always keep the one they can't work without.

Each tool_result, file read, and (once the window fills) stale history turn is compressed by the 4B model down to ~26% of its original size, tagged [REF:id]. This is where the trained model earns its keep: it knows a function signature from a debug line, so it protects identifiers, paths, and error strings while dropping the noise.

• Single-turn this is the smaller lever (a few % — most of a turn's cost is the fixed prefix). Its power shows up across a session — see below.
• Every compressed segment is recoverable: the agent calls read_original / expand_context to pull back the exact untouched bytes, locally and instantly.

Turns beyond the recent window are summarized once the context fills up, so a long session stays inside the model's context window instead of overflowing (or forcing an aggressive client-side compaction that drops detail).

This is the part a single-turn benchmark hides. In a real multi-turn session the two levers grow at different rates, and together they compound.

We ran the same read-only "find the bug" task for 5 consecutive turns in one Claude Code session (Sonnet, GPU model). Important: in this A/B the tool filter was left on for both sides, so the delta below isolates content compression alone — it does not include the tool-schema saving.

Content compression only (tool filter on both sides):

But that's only one of the three levers. The real "do I use Paritok or not?" comparison must add the tool filter back — and without Paritok the agent sends the entire ~29K tool-schema block every turn, not the filtered ~8K. Folding that ~21K/turn back onto the no-Paritok side:

Full stack (filter + compression vs. no Paritok at all):

So the 4.6% / 22.2% above is the floor (content only). Against a real no-Paritok baseline it's ~25% on turn 1, past ~39% by turn 5 — because the tool filter saves a fixed ~21K every turn on top of the compounding content compression.

Why it grows: every file you read stays in history and is re-sent (cache-read) every subsequent turn — so content compression keeps paying off turn after turn, while the tool filter adds a fixed cut on top.

• Content compression → quadratic. Cumulative saving ≈ 3,350 × N² — each turn's compressed reads keep paying off on every later turn.
• Tool filter → linear. Cumulative saving ≈ 21,000 × N — a fixed block saved every turn.
• Crossover ≈ turn 6: early on the tool filter dominates; past ~turn 6 content compression overtakes it and the gap widens.

Plugging these formulas into a range of N (baseline ~96,500 tokens/turn), capped at a ~200K context budget (typical Sonnet-tier configuration; larger-context models like Opus 1M push the flatten point out proportionally):

_{Capped at a ~200K context budget. The quadratic only holds while history is still growing. Once the accumulated context fills the configured budget (around turn ~8–12 here on Sonnet-tier ~200K), client-side compaction holds it flat, per-turn content saving stops growing (freezes at ~48K/turn), and % saved plateaus toward the ~72% default ceiling instead of diverging. Turns 1–5 match the measured tables above; later rows are in-window projections.}

Ceilings shift with deployment shape:

_{The projection above uses the default ceiling. MCP-heavy setups earn a bigger per-turn tool-filter cut; context-saturated sessions get an even larger effective saving because they're now compared against a compacted, information-lossy baseline.}

Honest cap: the quadratic doesn't run forever — whatever session context budget you configure bounds it. In practice a ~200K budget (typical for Sonnet-tier deployments) makes the curve flatten around turn ~12–20 as the window fills; larger-context models like Opus 1M push the flatten point out proportionally. But that ceiling is itself a feature: because each turn's prefix is smaller, the agent fits more turns before hitting the budget — Paritok effectively buys back context length.

Same budget, more room to think. Because each turn's prefix is smaller, the agent fits far more turns in the same window before compaction kicks in.

That's roughly 3× the runway on any budget, giving the agent more turns to think and finish the task before hitting the wall.

One line: use more, save more. Compression frees up the window and lets the agent go deeper and longer in the same session. Strongest on long, multi-turn, read-heavy work (auditing, Q&A over a big codebase, long debugging sessions).

The 74% figure is Paritok's content compression rate — file reads, tool output, and history shrink to ~26% of their size. End-to-end savings compound with session length.

Per-session cost at Claude Sonnet input pricing ($3 / M input tokens):

Annual savings by team size (120 turns/dev/day, ~54% average end-to-end):

_{Default ~40-tool config on Claude Sonnet. MCP-heavy setups (70+ tools) push savings to ~78%; context-saturated sessions past 85%. Use the interactive calculator at paritok.com for your specific team size, LLM, and workload — it plugs in Claude Opus ($5/M), GPT-5 ($10/M), Haiku ($1/M), Gemini, or any provider you use.}

Install and running in minutes on your own hardware. No vendor lock-in.

Self-host (Apache 2.0): Free forever. The gateway, the 4B model, and every training script ship open. Run on your own hardware. No telemetry, no key, no fees.

Hosted GPU (paritok.com): $0.30 per 1M tokens processed. No GPU required. Free through Oct. 1 as we launch, with full pricing kicking in October 1st.

Everything ships in the PyPI package — you do not need to git clone the repo. In a fresh environment (with Ollama installed):

pip install "paritok[proxy]"     # the gateway + CLI
paritok up                       # pulls the model if missing, then starts the proxy

paritok up checks Ollama, ollama pulls paritok/paritok-4b-v1 and tags it as the local paritok-4b-v1 if it isn't already there (~2.5GB, first run only), then serves on port 8080.

Leave that terminal running. The proxy is a foreground server — every agent request flows through it, so it must stay up for the whole session. Open a separate terminal for the next step.

In the shell that launches your agent:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080   # then start Claude Code / Cursor / …

No config file is needed — paritok up and paritok proxy run on built-in defaults. Run paritok init to drop a starter paritok.yaml only if you want to tweak settings.

q4 vs full precision. paritok up uses the q4 model (:latest, ~2.5GB) by default. For full precision, run paritok up --registry-model paritok/paritok-4b-v1:f16 (~8GB). If you already ollama pulled either variant, up auto-detects which one you have — no re-download.

pip install "paritok[proxy]"
# or, from a clone of this repo:
pip install -e ".[proxy]"

Enable the tool-schema filter (recommended — it's the biggest single-turn lever):

pip install "paritok[toolselect]"   # adds sentence-transformers (CPU-only)

First request warms up the embedding model (~10–15s — downloads bge-small once, then caches locally). Every request after is instant (~15 ms).

One boolean in paritok.yaml decides where compression runs:

use_gpu_server: false   # ← the only switch that matters

Option A — self-host (false, default). Run the open 4B model on your own machine. No key, nothing leaves your box. Simplest is Ollama:

ollama pull paritok/paritok-4b-v1                # one-time, ~2.5GB
ollama cp   paritok/paritok-4b-v1 paritok-4b-v1  # tag it as the runtime name

Default paritok.yaml already points local_model at Ollama — nothing else to set.

pip install vllm
vllm serve Qwen/Qwen3-4B-Instruct-2507 \
  --enable-lora \
  --lora-modules paritok-4b-v1=paritok/paritok-4b-v1 \
  --max-lora-rank 32 \
  --port 8000

Option B — Paritok GPU server (true). No GPU required. Create an API key at paritok.com → dashboard → API keys, then:

use_gpu_server: true
gpu_server:
  api_key: "pk_live_..."   # or: export PARITOK_API_KEY=pk_live_...

paritok proxy --port 8080 --config-file paritok.yaml

Keep this terminal open — the proxy must stay running for the whole session; run your agent from a separate terminal. On startup it checks the backend and warns (never aborts) if it can't reach one.

Set the base URL in the shell that launches your agent, then start the agent:

# macOS / Linux
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080   # Claude Code
export OPENAI_BASE_URL=http://127.0.0.1:8080      # Cursor / OpenAI-SDK agents

# Windows PowerShell
$env:ANTHROPIC_BASE_URL = "http://127.0.0.1:8080"
$env:OPENAI_BASE_URL    = "http://127.0.0.1:8080"

Keep your real provider API key set as usual (ANTHROPIC_API_KEY / OPENAI_API_KEY) — the proxy only rewrites the request body and forwards your headers upstream. Compressed prompts go upstream; original responses come back unchanged.

Codex is the exception — it does not read OPENAI_BASE_URL. The Codex CLI takes its endpoint from ~/.codex/config.toml. See Codex setup.

The gateway is independent of the upstream LLM — point the OpenAI path at any OpenAI-Chat-Completions-compatible endpoint with --openai-url:

paritok proxy --openai-url https://api.groq.com/openai        # Groq
paritok proxy --openai-url https://openrouter.ai/api/v1       # OpenRouter
paritok proxy --openai-url https://api.orcarouter.ai/v1       # OrcaRouter

Then point your OpenAI-SDK agent at the proxy (OPENAI_BASE_URL=http://127.0.0.1:8080), set OPENAI_API_KEY to the provider's key, and use that provider's model names.

Codex ignores OPENAI_BASE_URL — it only reads its endpoint from ~/.codex/config.toml, so Paritok writes that file for you:

codex:
  enabled: true            # paritok writes ~/.codex/config.toml on `paritok up`
  model: gpt-5             # any model your key can call
  api_key: "sk-..."        # your OpenAI key (or leave "" to use env OPENAI_API_KEY)

paritok up     # writes ~/.codex/config.toml (backs up any existing one)
codex          # in another shell — now routed through Paritok

Codex custom providers only speak the responses wire protocol, which the proxy serves at /v1/responses.

curl http://127.0.0.1:8080/health   # {"status":"ok","version":"..."}
curl http://127.0.0.1:8080/stats    # live compression totals

/stats reports cumulative savings across the session:

{
  "total_requests": 42,
  "input_tokens_original": 512340,
  "input_tokens_compressed": 138221,
  "compression_ratio": 0.27,
  "tokens_saved": 374119,
  "estimated_cost_saved_usd": "$1.01"
}

These numbers are scoped to what Paritok actually intervenes in — the content it compresses plus the tool schemas it stubs. Everything it can't affect (your system prompt, the model's output) is excluded. estimated_cost_saved_usd is cache-aware: compressed content is priced at the full base rate (new input each turn), while the frozen tool block is priced as a prompt-cache hit after turn one — counting it at full list price would massively overstate savings.

import anthropic
import paritok

client = paritok.ParitokClient(anthropic.Anthropic())
resp = client.messages.create(
    model="claude-sonnet-4-20250514", max_tokens=4096, messages=[...]
)
print(resp._paritok_savings.saved_tokens, resp._paritok_savings.ratio)

Just want compressed text — no agent, no proxy (e.g. a RAG / retrieval pipeline)? Call the hosted endpoint directly with your Paritok API key. Unlike the proxy it returns clean compressed text with no [REF:] tag — that tag belongs to the agentic expand/recall loop, which direct callers don't use — plus the token counts:

curl -sX POST https://www.paritok.com/api/compress \
  -H "Authorization: Bearer pk_live_..." \
  -H "Content-Type: application/json" \
  -d '{
        "content": "…text to compress…",
        "query": "what the compression should keep in view",
        "upstream_model": "claude-sonnet-4"
      }'

{
  "compressed": "…clean compressed text, no [REF:]…",
  "gpu_available": true,
  "original_tokens": 2903,
  "compressed_tokens": 88
}

query is optional but recommended — it's the intent the compressor keeps (it drops what the intent doesn't name). If the GPU backend is unavailable it degrades safely: gpu_available: false and compressed echoes your original unchanged (nothing lost, just uncompressed).

Paritok is most useful when:

• Your sessions are long and multi-turn. Savings compound from ~25% at turn 1 to 60%+ by turn 20.
• You're read-heavy or mixed: auditing, Q&A, debugging over a large codebase.
• You have many MCP tools (40+). The tool filter alone drops ~21K tokens per turn.
• You're paying premium input rates (Claude Sonnet $3/M, Opus $5/M, GPT-5 $10/M).
• You want to fit deeper sessions in the same context window and avoid forced client-side compaction.

The gateway's content compression is powered by Paritok-4B-v1, the first open-source compression model trained end-to-end on real coding-agent trajectories. This section is the engine spec — you don't need it to run the gateway, but it's why the compression keeps what matters.

• 🎨 Code-native. Trained on real coding-agent trajectories (file_read, bash_command, log_output, …). It knows what an import statement is worth vs a debug line, so it protects function names, paths, and error strings while compressing.
• 📋 Extractive — audited, not asserted. It selects spans instead of rewriting them: 96.2% of the identifiers, paths, and numbers it emits on held-out SWE-bench Lite are copied straight from the input, so an exact-match edit against compressed context still applies.
• 🚀 Compresses each segment to 25.7% of original — 2× harder than gpt-4.1-mini (50.2% CR) and 2.4× harder than gpt-5 (61.9% CR).
• 🎯 Retains 86.5% of full-context solve quality on SWE-bench Lite (89.3% fed the line-numbered input it was trained for, where the paired difference against uncompressed is not significant, p = 0.079) — matching gpt-4.1-mini as compressor at less than half the token spend.
• 🪶 Small & self-hostable — 4B LoRA adapter, bf16, runs on a single 24GB GPU. No SaaS, no lock-in, no per-token compressor fee.
• 🔓 Fully open — Apache 2.0 weights, reproducible data pipeline, real end-to-end SWE-bench numbers.

Real end-to-end evaluation. An agent scaffold receives its context through each compressor, then attempts to resolve the issue. Primary metric is quality retained (solve rate normalized to the uncompressed baseline).

_{¹ Quality retained = compressor solve rate ÷ uncompressed baseline solve rate, where solve rate is resolved ÷ all 300 instances — a patch that fails to apply counts as unresolved. Higher is better.}

_{The last row feeds the compressor cat -n line-numbered source, which is what real agents actually read (--line-numbers); it is the configuration we recommend. On that run the agent resolves 122/300 uncompressed and 109/300 compressed — 30 instances solved only without compression, 17 only with it. An exact McNemar test on those 47 discordant pairs gives p = 0.079: at 300 instances, compressing the context to roughly a quarter of its size does not significantly reduce the solve rate. The GPT rows are quoted against the raw-source row so all three compressors see the same input.}

The benchmark is a floor, not a ceiling. The 86.5% measures the raw 4B model with no recall enabled — compressed output fed straight to the agent. What you actually deploy is the gateway: every segment is tagged [REF:id] and the agent can call read_original to pull back the exact bytes at any time. Nothing is permanently discarded, so real-world deployment recovers quality the raw benchmark leaves on the table. We publish the raw-model number because it's the honest, reproducible floor.

To reproduce these SWE-bench Lite numbers yourself, see the eval_model/ folder — one command runs it end to end (pull from source → compress → score).

Compression is only safe for an editing agent if the model selects spans rather than rewriting them — a reworded function name breaks the next exact-match edit. That's a design commitment, so we measure it instead of asserting it: once over the training corpus, and again over output on data the model has never seen.

_{SWE-bench Lite is held out end to end — no instance of it appears in training — so the second row is the one that answers does the copy behavior survive off-distribution? There, the per-instance median token-copy rate is 97.0% and 213 of 300 instances sit at or above 95%. The corpus row rises to 98.3% once assistant_thinking is excluded, which is abstractive by rule (as are meta_action and long string literals). The residual is real rather than measurement noise — the teacher occasionally completes a name the input only partly contained (Exception type: FailedParse → tatsu.exceptions.FailedParse:), and the student learned that along with everything else. So the honest claim is substantially extractive — copy-first by construction, 96–98% copied in practice — alongside the narrower rule that does hold absolutely: retained code lines and error strings are copied verbatim. One caveat we state rather than round away: SWE-bench Lite instances are unseen, but some of its repositories are popular enough to also appear in the trajectories the corpus was distilled from, so this is unseen-instance, not unseen-repository, generalization.}

Both rows are reproducible:

# training corpus, by segment kind
python eval/extractiveness.py

# held out — re-derived offline from the cache a finished run leaves behind
# (no GPU, no API calls), alongside the per-instance compression distribution
python eval_model/audit_swebench.py \
    --cache eval_model/_work/instances_chunk3000_paritok-4b-v1_latest_ln.jsonl

_{extractiveness.py reads the distilled corpus under update/, which isn't committed — regenerate it with the data pipeline. audit_swebench.py needs only the resumable cache your own eval_model/run.py run already wrote.}

• 🎯 Paritok-4B-v2. Next-gen pipeline pushing compression under 20% while closing the gap to uncompressed solve rate.
• 📈 Frontier-scale backbones. Larger models (10B+) for multi-day sessions with 100K+ token histories.
• 🌍 Multi-language expansion. First-class TypeScript, Rust, Go, Java, C++, Kotlin — v1 is Python-heavy but the architecture is language-agnostic.
• 🔌 Native integrations. Drop-in mcp add paritok plugin for Claude Code and Cursor.
• ⚙️ Adaptive compression. Per-segment auto-selection of aggressiveness based on age, kind, and downstream intent — no manual tuning.

Paritok is built by two engineers — no big lab, no external funding, just months of GPU budget and eval iteration.

• Jiayu Shi — Architecture of the 4B on-device compression model, data pipeline (cleaning + construction), reward design and fine-tuning.
• Luzhuo Chen — Inference acceleration, full-stack architecture and deployment, evaluation framework, high-concurrency gateway optimization.

Reach us: paritok9@gmail.com · X @Paritok

Cite the paper for the method and results:

@misc{paritok4b,
  title         = {Paritok-4B: Intent-Conditioned Context Compression for Coding Agents},
  author        = {Shi, Jiayu and Chen, Luzhuo},
  year          = {2026},
  eprint        = {2608.24188},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  doi           = {10.48550/arXiv.2608.24188},
  url           = {https://arxiv.org/abs/2608.24188},
}

Cite the companion paper for end-to-end cost — what a compression rate does and does not translate into over a full agent session:

@misc{paritok-cost-attribution,
  title     = {An Empirical Cost Attribution of Context-Compression Gateways in
               Multi-Turn Coding Agents: Why Compression Rate Is Not the End-to-End Saving},
  author    = {Chen, Luzhuo and Shi, Jiayu},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.22043550},
  url       = {https://doi.org/10.5281/zenodo.22043550},
}

Cite the software for this release — the gateway, the weights, and the training and evaluation code:

@software{paritok-4b-v1,
  title   = {Paritok-4B-v1: An Open-Source Compression Model for AI Coding Agents},
  author  = {Shi, Jiayu and Chen, Luzhuo},
  year    = {2026},
  version = {4B-v1},
  license = {Apache-2.0},
  url     = {https://github.com/Paritok-official/paritok-4b-v1},
}

Apache 2.0 — see LICENSE. The base model, Qwen3-4B-Instruct-2507, is released under its own license.

• 👥 Join the community → Discord
• 🐛 Bug reports & feature requests → GitHub Issues
• 💭 Discussion → 🤗 HF Model discussions
• 📧 Contact → paritok9@gmail.com