alexgreensh/token-optimizer

English · 한국어 · 简体中文 · 日本語

It runs automatically in the background. You keep working as usual. Run the audit when you want the full picture.

Token Optimizer cuts the tokens your AI coding assistant wastes, keeps your work alive across sessions and compactions, and shows you where every dollar went on a live dashboard. Most of it runs automatically. You install it, run the audit once, and the hooks do the rest.

Why not just use Headroom or RTK? They compress command output, which covers 15-25% of your context. Token Optimizer covers that plus the other 75%: bloated configs, unused skills, stale memory, compaction loss, model misrouting, behavioral waste. Every saving is cache-safe and measured. The dashboard updates after every session, automatically.

Works on the AI coding tools below, with a Devin output-filter beta (manual skill, or install.sh --devin --auto for automatic compression):

Runs automatically, every session, you do nothing:

• 🔄 Smart Compaction: checkpoints before auto-compact, restores after
• 🗄️ Session Continuity: cross-session hints, cold-resume, checkpoint scoring
• 📦 Active Compression: 9 features, all on by default (delta diffs, skeletons, bash/search compression, lean-output nudges, quality nudges, loop detection, activity mode, decision extraction)
• 📊 Quality Scoring: 7 signals, real-time, letter grades S–F
• 🗃️ Session Database: SQLite, 15 tables, full audit trail, zero network
• 🔍 Progressive Disclosure: large outputs archived, expand on demand
• 🧠 Context Intel Digest: post-compaction re-orientation without re-reads
• 🔀 Model Routing Nudges: steers to the right tier for the task

When you ask for it:

• 🩺 /token-optimizer: full audit with guided fixes
• 📈 /token-coach: 30-day trend analysis with specific fixes
• ⚡ quick: 10-second health check
• 🎯 route: the model and effort this task actually needs, before you spend
• 🔧 doctor: installation check
• 💰 savings: dollar savings report
• 📋 report: per-component token breakdown
• 🌐 dashboard: open the full dashboard
• 📝 memory-review: MEMORY.md structural audit
• 📂 expand: retrieve archived tool result
• 🔙 resume-lean: reopen a cold session

Install, run /token-optimizer once, everything else runs automatically.

_{A 30-second preview. Watch the full film with sound (50 seconds).}

Claude Code now lets a plugin add its own piece of interface, called a mod. Token Optimizer uses one to put your session's health right above the prompt: a quality grade, how full the context is, how long the cache stays warm, and both usage limits. One plain sentence tells you what needs attention, and one button fixes it. Clawd, the small orange mascot, acts out what the session is doing, so you can tell at a glance whether Claude is reading, editing, waiting for your OK, or done.

It comes with the plugin, nothing extra to install. Everything the bar does.

Claude Code (recommended):

/plugin marketplace add alexgreensh/token-optimizer
/plugin install token-optimizer@alexgreensh-token-optimizer

Then in Claude Code: /token-optimizer

Enable auto-update after installing. Claude Code ships third-party marketplaces with auto-update off by default. /plugin → Marketplaces tab → select alexgreensh-token-optimizer → Enable auto-update. One-time, 10 seconds.

After install, run /token-optimizer once to set up hooks. From there, everything runs automatically: compression, checkpoints, quality scoring, dashboard updates. You don't need to run any command again unless you want an audit.

Claude Code cloud sessions (claude.ai/code). A cloud session runs in a fresh container and never reads the plugins installed on your machine, so the two commands above are not enough there (/plugin itself is not available in cloud sessions). Either enable Token Optimizer for your claude.ai account (Desktop app → Customize → plugins), or commit this to the repo's .claude/settings.json so every cloud session on that repo installs it at start:

{
  "extraKnownMarketplaces": {
    "alexgreensh-token-optimizer": { "source": { "source": "github", "repo": "alexgreensh/token-optimizer" } }
  },
  "enabledPlugins": { "token-optimizer@alexgreensh-token-optimizer": true }
}

Hooks, compression and redaction behave the same inside the container. Each cloud session starts with an empty state directory, so the dashboard and audit history there cover that session only.

codex plugin marketplace add alexgreensh/token-optimizer

{ "$schema": "https://opencode.ai/config.json", "plugins": ["token-optimizer-opencode"] }

openclaw plugins install github:alexgreensh/token-optimizer

git clone https://github.com/alexgreensh/token-optimizer.git
token-optimizer/install.sh --hermes

pi install git:github.com/alexgreensh/token-optimizer

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --copilot

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --cursor

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --antigravity

git clone --depth 1 https://github.com/alexgreensh/token-optimizer.git
cd token-optimizer
bash install.sh --grok

tmp="$(mktemp -d)"
release_json="$(curl -fsSL https://api.github.com/repos/alexgreensh/token-optimizer/releases/latest)"
tag="$(python3 -c 'import json,sys; print(json.load(sys.stdin)["tag_name"])' <<<"$release_json")"
git clone --branch "$tag" --depth 1 https://github.com/alexgreensh/token-optimizer.git ~/.claude/token-optimizer
bash ~/.claude/token-optimizer/install.sh
rm -rf "$tmp"

sed -i 's/\r$//' ~/.claude/token-optimizer/install.sh
# already have the repo? re-normalize line endings in place:
git -C ~/.claude/token-optimizer add --renormalize . && git -C ~/.claude/token-optimizer checkout -- .

/plugin uninstall token-optimizer@alexgreensh-token-optimizer

Most token tools compress command output. That covers 15-25% of your context. The other 75% goes untouched.

Compression coverage. Headroom, RTK, and JFrog Boost compress bash and command output. Token Optimizer compresses eight surfaces of the output stack. Status: 🟢 supported, 🟡 partial, 🔴 not supported.

RTK and Boost reach the first surface. Headroom reaches the first and the third. Token Optimizer covers all eight, then keeps going into what happens around compression:

• Three kinds of waste, not one. Structural (bloated configs, unused skills, stale memory), runtime (verbose output, re-reads), and behavioral (model misrouting, cache expiry, retry loops). How each works →
• Savings survive compaction. Checkpoints before auto-compact, restores after. Without this, compression savings vanish the moment compaction fires.
• Measures whether it helped. Before/after token deltas, dollar savings across four pricing tiers, quality scores that track degradation. Not just "we compressed stuff."
• Zero baseline overhead. External process, no always-on instructions in your context, no MCP server, no dependencies, no telemetry.

These ten rows are the ones where Token Optimizer is the only 🟢 in the row: what survives beyond compression, and the waste no compressor looks for. Compression mechanics, cost and safety, and the two rows where we lose are behind the fold.

One HTML page, auto-regenerates after every session via the SessionEnd hook, no manual trigger needed. The bookmarkable URL http://localhost:24842/token-optimizer is opt-in: it only works after you run python3 skills/token-optimizer/scripts/measure.py setup-daemon to start the local server. Until then, open the file path that measure.py dashboard prints on the Dashboard: line.

Per-turn token breakdowns, cost across four pricing tiers, cache analysis with TTL mix and hit rate, quality scores overlaid on every session, subagent cost breakdown, savings tracker with metered and estimated tiers kept apart. Zero setup after install. Full dashboard docs →

Six levers, all automatic: tool-output compression, cross-turn dedup, delta-read, structure-map skeletons, checkpoint-restore, and model routing. What they save is tracked in three tiers.

Three tiers, labelled by how each is known (one real month, the author's last 30 days, all six levers on):

• Logged actions (~$126), metered. Every saving logged as it happened: a result archived, a repeat read skipped, a lean resume, a loop cut short. 2,733 events with before and after token counts.
• Repeat reads avoided (~$1,124), estimated. Content Token Optimizer removed would have been re-read on every later turn until the session's next real compaction, modelled per turn at that turn's cache-read rate and capped at the real compaction event.
• Other estimates (~$146), estimated. Lean session resumes, compressed sub-agent context, MCP output caps, loops prevented.

About ~$1,396 in API-equivalent value for the month, and 59.6M tokens never sent to the model, roughly 28% of the workload (150.8M spent, about 210.4M without Token Optimizer). Tokens are the primary figure; dollars depend on which models you run. Metered and estimated are never dressed up as one number. See the full methodology →

Based on 1,698 sessions over the last 30 days (snapshot 2026-09-14). Your number is your own: the dashboard computes the same report from your own sessions. See the methodology →

Nine features that actively reduce context, all on by default, all automatic, all toggleable from the dashboard or CLI.

Under the hood, PreToolUse hooks intercept every Read and Bash call before it enters your context. If the file was already read, only the diff comes back. If it's a code file re-read, a structural skeleton replaces the full content. If it's a CLI command, the output is compressed. PostToolUse hooks archive the full original to disk and log a compression event to SQLite. Nothing is lost, and everything is retrievable. You do nothing. The hooks handle it all.

Toggle from the dashboard Manage tab, CLI (measure.py v5 enable|disable <feature>), or env vars. The v5 verb is a legacy command name that controls current features.

Read how each feature works →

# Add a read-only command to an existing compression handler.
# handler must be one of the built-in pattern names (pytest, lint, ls, etc.).
[filters.add]
my-tests = { command = "cargo test", handler = "pytest" }
my-lint = { command = "ruff check", handler = "lint" }

# Exclude commands from compression (glob patterns supported).
[filters.exclude]
commands = ["git status", "ls -la"]

python3 measure.py compression-stats --days 30

Everything Token Optimizer does is backed by two local SQLite databases. Nothing leaves your machine. Zero network calls.

Per-session DB (~/.claude/token-optimizer/snapshots/session-store/<session>.db) holds 8 tables tracking file reads, tool outputs, command outputs, cached content, context intel events, activity log, decision log, and hint serves. WAL mode for concurrent read/write from hook processes. 50MB cap per session.

Trends DB (~/.claude/token-optimizer/snapshots/trends.db) holds 7 tables tracking session history, daily aggregates, skill/model/subagent usage, savings events, and compression events. Indexed by session UUID for O(log n) joins. This is what powers the dashboard, coach mode, and 30-day trend analysis.

Every compression event, every saving, every quality measurement is a row you can query. The dashboard is a read-only view of this data. measure.py compression-stats is a SQL query. Your data is your data.

Large tool results (>4KB) are archived to disk and replaced with a short preview plus a retrieval pointer. The full output survives compaction. When the model needs it, it pulls the original via expand, with no command re-run and no lost output.

Recovery is not compressed a second time. The optimizer's own measure.py expand <id> command (including --session <id>) and known original-recovery tools pass through without a new archive, dedup replacement or re-fetch denial. Ordinary memory search and document retrieval keep their existing behavior.

This isn't just storage. The system tracks how many results were archived vs re-expanded, so you can see the net tokens that stayed collapsed. Re-expansions are netted out of the savings total, so you only count what actually stayed compressed.

python3 measure.py expand --list          # List archived tool results
python3 measure.py expand <tool-use-id>   # Retrieve a specific result
python3 measure.py expand --search <query>  # Search archived results by keyword/lineage

expand --search runs a full-text search (FTS5 with a LIKE fallback) over archived tool outputs, including the full response body, tool name, command, source file path, language, and archive origin. Each match prints the key, lineage, and a snippet. Use expand <key> to retrieve the full content.

Some MCP tools are documented to return large verbatim payloads (source fetchers, doc retrievers) where a metadata preview defeats the point. Allowlist those so their full result reaches context instead of being replaced (it's still archived, so expand works). Set a comma-separated list of fnmatch globs matched against the full tool name:

TOKEN_OPTIMIZER_ARCHIVE_EXEMPT_TOOLS="mcp__context7__*,*github_repository_file"

Default is empty — nothing is exempt unless you opt a tool in.

Compression matters, but the most important thing Token Optimizer does is keep your work alive across sessions and compactions, automatically.

When you end a session, Token Optimizer checkpoints your state to SQLite: active task, key decisions, modified files, git branch, recent reads. When you start a new session, it scores all recent checkpoints against your prompt and surfaces the most relevant one as a hint. You resume with context, not from zero.

What happens automatically:

• Cross-session hints: relevance-scored checkpoints surface when you start a new session. The hint includes the prior session's task, decisions, files, and branch. All fenced as RECOVERED DATA, never instructions.
• Cold-resume-lean: reopen a stale session without paying the full transcript cost. Token Optimizer reconstructs a lean context from its checkpoint. No LLM call, no full-transcript cold-resume. Token-free reconstruction from SQLite.
• Hint-follow measurement: when a continuity hint surfaces file paths and the model subsequently reads one, Token Optimizer credits an avoided exploratory search. Measured causality, not a guess.

python3 measure.py resume-lean                    # list reopenable cold sessions
python3 measure.py resume-lean <#|session_id> --print  # emit lean context block

Compression savings only stick if your session survives compaction. Session continuity is what makes that happen.

When auto-compact fires, 60-70% of your conversation vanishes. Decisions, error-fix sequences, agent state, all gone.

Token Optimizer checkpoints your session before compaction and restores what the summary dropped, automatically, via hooks. It also injects a context intel digest: heuristic summaries of large tool outputs the model already processed (file paths touched, errors seen, line counts). After compaction, the model knows what it saw without re-reading everything.

Activity mode detection classifies your session in real time (code, debug, review, infra, general) using the last 10 tool calls. The mode feeds into compaction guidance so PRESERVE/DROP priorities adapt to what you're doing right now, not a generic heuristic.

Decision extraction captures decision statements from tool outputs as they happen and stores them in the session DB. At compaction time, these are injected as CRITICAL DECISIONS the summary must preserve verbatim. Capped at 10 per session.

Compression savings only stick if your session survives compaction. Saving tokens on git status doesn't help if the next auto-compact wipes out the decision that made you run it.

python3 measure.py setup-smart-compact    # checkpoint + restore hooks

Output tokens are the most expensive part of your session. They cost 5x more than input tokens on Opus and are billed per generation, not per cache read. A verbose response to a simple question burns dollars you never needed to spend.

Token Optimizer handles this automatically with lean-output nudges. When your context fills past 25%, a short nudge tells the model to reason deeply but keep visible output lean. Fill is the only trigger; context quality is not a condition. The reduction is estimated at 10-15% and is not counterfactually measured, so it is reported in the estimated tier, separately from metered savings.

How it works:

• The nudge is injected as additionalContext, never modifying the existing prefix, so your cache stays intact
• It only fires when context is filling up, not when you have plenty of room
• The model still thinks through the problem; it just produces a more concise visible answer
• No on/off switch today; tune the trigger point with TOKEN_OPTIMIZER_VERBOSITY_MIN_FILL (default 25)

This is one of the 9 active compression features, and it's the one that saves on the output side, where tokens cost the most.

The quality score tracks two things: Resource Health (how close you are to the degradation cliff) and Session Efficiency (whether your tokens are doing useful work). Letter grades from S to F make triage instant.

As context fills, quality drops. MRCR falls from 93% to 76% between 256K and 1M context. Your AI gets measurably dumber as the window fills. The quality score shows you exactly when that happens.

The status bar shifts color as quality degrades: green, yellow, orange, red. When quality drops below 75, session duration appears as a warning. Running subagents show with their model and elapsed time.

python3 measure.py setup-quality-bar      # one-time install

Read how scoring works →

python3 measure.py setup-quality-bar --uninstall

Using the Claude desktop app? The status bar comes with the Token Optimizer plugin: once it is installed, the bar appears above your prompt. Nothing extra to install.

It needs Claude Code 2.1.287 or newer and Anthropic's mods feature. Anthropic can switch mods off remotely; if that happens nothing breaks, the bar just doesn't appear. The terminal keeps its existing status line. VS Code is not supported. To hide it, add "TOKEN_OPTIMIZER_STATUS_BAR": "0" to the env block of ~/.claude/settings.json; "TOKEN_OPTIMIZER_STATUS_BAR_ANIMATE": "0" keeps Clawd still.

• One sentence, one button: the most urgent thing about your session, with at most one button to act on it.
• Five marks: quality grade, context fill, cache countdown, 5-hour limit and weekly limit, each with its label. Hover any mark for a card with the detail.
• The arrow beside Clawd unfolds one more line: branch, session time, tool calls, compactions (when there are any), when the last checkpoint was saved (any kind, or a relevant one from an earlier session), and how many tokens Token Optimizer saved you in the last 30 days, the same total the dashboard shows. The last thing on that line is a Full dashboard link that regenerates the dashboard and opens it in your browser.
• Clawd acts out the session: thinking, reading, typing, subagents, waiting for your permission, writing, compacting, done, stopped, API error, cold cache, napping.
• Clean up: compacts with Token Optimizer's guidance.
• Start fresh: asks for a second click, then saves a checkpoint, clears, and hands the checkpoint to your first message.
• Keep warm: a manual, one-click cache refresh. It is offered only while the cache is still warm, never runs on its own, and uses a small amount of your usage. Afterwards it tells you how many tokens it re-read from the cache.
• Two sizes: the – button beside the arrow shrinks the band to one line (tiny Clawd, the marks, and whatever needs attention) and + brings it back. The arrow still unfolds the details row under it. The choice is remembered across sessions and restarts; TOKEN_OPTIMIZER_STATUS_BAR_SIZE=slim sets the starting size and makes the terminal status line print one line too.

Point at the bar and Clawd looks over. The cache countdown is an estimate, because Claude does not publish the exact expiry: 1 hour on Claude plans, 5 minutes on the API, measured from the session itself when possible.

> /token-coach

Tell it your goal. Get back specific, prioritized fixes with exact token savings. It reads 30 days of your session data and surfaces what no single session can show: quality drifting down, sessions getting longer, cache hit rates falling, cost per session climbing.

Every insight is grounded in your actual numbers. "Your short sessions score 68 vs 60 for long ones" hits differently than "consider shorter sessions." Coach Mode also identifies project-level optimization opportunities (skills you never use, MCP servers that load eagerly, CLAUDE.md patterns that break your cache) and teaches you how to fix them so future sessions start leaner.

Read about Coach Mode →

python3 measure.py keepwarm-enable          # opt in (API billing only)
python3 measure.py keepwarm-report            # net savings, spend, tripwire state
python3 measure.py keepwarm-disable           # opt out any time

python3 measure.py subagent-cache status    # state, the verdict and why, the advice, estimated net
python3 measure.py subagent-cache enable    # take the recommendation, whatever the estimate says
python3 measure.py subagent-cache disable   # undo (only what Token Optimizer set; final)
TOKEN_OPTIMIZER_SUBAGENT_CACHE_1H=1           # opt in to the automatic write + tripwire
TOKEN_OPTIMIZER_SUBAGENT_CACHE_1H=0           # never: opt out permanently

python3 measure.py recommendations            # the daily record: state, numbers, the exact command
python3 measure.py recommendations --json     # machine-readable
python3 measure.py recommendations refresh    # re-measure now (bounded; locks against a second run)

python3 measure.py inject-routing --dry-run   # Preview
python3 measure.py inject-routing              # Inject

PolyForm Noncommercial 1.0.0, plus a written small business permission. Source-available.

Go for it. Full source, runs locally, no license purchase needed. That includes your own open-source side projects. What counts is who the work is for, not where it runs: work for your employer is your employer's use, even from home.

No-cost commercial license, in writing: LICENSE-SMALL-BUSINESS.md. Both limits apply to your whole company (including any parent or sister companies), not to the team or the number of people using Token Optimizer. A small team inside a larger company needs a commercial license.

Your past use is totally fine. You get 32 days after you stop qualifying to sort out a commercial license.

Contact Alex Greenshpun or me@alexgreenshpun.com. Terms are reasonable and scale with company size.

Created by Alex Greenshpun.