lynote-ai/ai-text-detector

An explainable, cautious AI-generated text risk analyzer for coding agents and local workflows.

简体中文 · Install · Evaluation · Use Cases · Detector API

🚀 New: the Lynote AI Detector API is live. Need AI-text detection inside your own app, LMS, or moderation pipeline? Call Lynote's production detector with one REST request — no model to train or host. Get an API key → lynote.ai/ai-detector-api

Other Quality Projects
AI Humanize Text:https://github.com/lynote-ai/humanize-text
AI Image Detector:https://github.com/lynote-ai/ai-image-detector

Two tiers in this repo. This page documents the lightweight local analyzer — an explainable, no-network CLI/skill for fast triage. The production two-level model (trained, multilingual, served) lives in detection-model-pipeline/; see How the production model works.

Most AI text detectors are either overconfident, opaque, or awkward to embed inside agent workflows.

ai-detector-skill takes the opposite approach:

• explainable weighted signals instead of hidden model claims
• a local CLI and Python API that are easy to script
• a short-text guardrail that refuses to overstate weak evidence
• skill-ready packaging for Codex, Claude Code, and other repo-aware agents
• reproducible benchmark and dataset evaluation scripts

If you want a triage tool that stays cautious and leaves room for human review, this repo is built for that.

The heavyweight detector behind Lynote is a two-level model that, in one forward pass, labels a document (human / ai / mixed) and every sentence (human / ai / paraphrased). It is trained, not rule-based — the full pipeline (data prep → train → calibrate → eval → serve) lives in detection-model-pipeline/.

flowchart LR
    csv["detection-log export (CSV)"] --> prep["prepare · dedup + soft labels"]
    prep --> train["train · joint doc + sentence heads"]
    hub["public backbone · XLM-R / ModernBERT"] --> train
    train --> cal["calibrate · temperature scaling"]
    train --> eval["eval · two-level metrics"]
    cal --> serve["serve · FastAPI, {code,msg,data}"]

• One model, two levels. Document and sentence heads share a backbone; L = L_doc + α·L_sent. Differentiable sentence-probability statistics feed the document head — which is what makes the mixed class trainable.
• Multilingual backbones. XLM-R-large (default) or ModernBERT-base, optimized for en / zh / pt / es.
• Calibrated, not overconfident. Temperature scaling softens probabilities without changing predictions; confidence bands reject < 0.33 ≤ low < 0.6 ≤ medium < 0.8 ≤ high.
• Deterministic & bounded. Hash-pinned splits, synthetic demo data, and make check gates (manifest / demo / docs / tests). No weights, datasets, or service logs are shipped, and no accuracy claims are made.

Full method: MODEL.md · ARCHITECTURE.md · DATA_CONSTRUCTION.md · API.md

The local analyzer here is built for cautious, offline triage, and the production model above ships as a pipeline, not weights. If you need production-grade detection inside your own product, the Lynote AI Detector API serves Lynote's production detector through a single endpoint:

• One synchronous REST call — send text, get structured JSON back with an overall classification, confidence, and sentence-level signals.
• Covers major models — detects patterns associated with ChatGPT, Gemini, Claude, DeepSeek, and more.
• Multilingual — the same endpoint handles multiple languages.
• Server-side keys — Bearer-token auth, with separate credentials per environment.
• Simple billing — 1 credit per input word (Chinese is counted per character: 1 character = 1 word).

curl https://api.lynote.ai/v1/detect \
  -H "Authorization: Bearer $LYNOTE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "The rapid adoption of large language models has reshaped how teams write."}'

Plans, rate limits, and full docs: lynote.ai/ai-detector-api. As with any detector, treat results as a signal for human review, not proof of authorship.

pip install -e .
ai-detect examples/sample_ai_like.txt --json

Or bootstrap a local environment:

bash scripts/setup.sh

Example output:

Conclusion: AI-like signals are present, but this medium-confidence score is a risk estimate rather than proof.
Score: 84/100
Confidence: medium
Verdict: high_ai_likelihood
Words analyzed: 256

• score: 0-100 AI-like writing risk estimate
• verdict: insufficient_text, low_ai_likelihood, mixed_or_uncertain, or high_ai_likelihood
• confidence: currently low or medium
• signals: strongest weighted evidence signals
• caveats: warnings that stay attached to the result
• next_steps: practical follow-up actions when useful

This is designed to help an agent say:

• "AI-like signals are present."
• "The sample is too short for a meaningful estimate."
• "This should be reviewed against known writing samples."

Not:

• "This was definitely written by AI."
• "The detector proves misconduct."

ai-detect examples/sample_ai_like.txt
cat essay.txt | ai-detect --json
python scripts/detect.py examples/sample_human_like.txt --json

from aidetect import analyze_text

text = open("essay.txt", encoding="utf-8").read()
result = analyze_text(text)

print(result.score, result.confidence, result.verdict)
for signal in result.strongest_signals():
    print(signal.name, signal.note)

cp -R ai-detector-skill "$CODEX_HOME/skills/"

Use the root SKILL.md as the portable skill definition, and keep AGENTS.md at the repo root for repo-aware agents.

We ran a reproducible evaluation on the public HC3 dataset, using the English finance, medicine, and open_qa subsets with the first 100 rows from each subset.

Snapshot of the lightweight local analyzer on that slice (not the production two-level model):

• Human mean score: 5.4
• AI mean score: 18.4
• Mean separation: 13.0 points
• Human coverage: 0.427
• AI coverage: 0.920
• Covered accuracy at score >= 45: 0.317

What that means in practice:

• the detector separates human and AI answers on average, but only weakly on HC3
• the short-text guardrail is doing useful work, especially on shorter human answers
• the current thresholds are conservative, which keeps false confidence down but also lowers recall
• this tool works better as triage + explanation than as a stand-alone classifier

See the full report in docs/HC3_EVALUATION.md.

Reproduce it with:

make eval-hc3

A teacher receives a polished 400-word reflection and wants a cautious signal before doing manual review.

Suggested workflow:

1. Run ai-detect submission.txt --json.
2. Read the strongest signals and caveats.
3. Compare the passage with known writing samples before making any judgment.

An editor wants to spot formulaic product reviews or guest posts before spending time on manual edits.

Why it fits:

• medium-length prose works better than short snippets
• explainable signals help justify why a draft feels templated

A moderation team wants to sort suspicious long-form posts into a manual review queue, not auto-remove them.

Why it fits:

• the tool is conservative by design
• it helps with prioritization more than enforcement

A team compares human drafts and AI-assisted drafts to see where language starts sounding too generic.

Why it fits:

• the score is useful as a relative signal across versions
• strongest signals can guide rewriting

• disciplinary decisions about a named student or employee
• treating a single score as proof of cheating or fraud
• very short samples under about 80 words
• high-stakes authorship disputes without known-sample comparison

detection-model-pipeline/   # production two-level model — prepare → train → calibrate → eval → serve
├── code/                    # pipeline stages (i..m) + harness validators
├── model_entity/            # joint model: ModernBERT + XLM-R backbones
├── data/demos/              # synthetic demo data (hash-pinned)
└── docs/                    # ARCHITECTURE / MODEL / DATA_CONSTRUCTION / API

ai-detector-skill/           # lightweight local analyzer (this page)
├── SKILL.md
├── scripts/
│   ├── detect.py
│   ├── setup.sh
│   ├── benchmark.py
│   └── evaluate_hc3.py
├── references/
│   └── api-reference.md
├── assets/
│   ├── hero.svg
│   ├── score-bands.svg
│   ├── workflow.svg
│   └── templates/
│       └── report.md
├── src/aidetect/
├── tests/
├── AGENTS.md
└── README.md

make test
make demo
make benchmark
make eval-hc3

GitHub Actions automatically runs:

• make test on Python 3.9, 3.11, and 3.13
• make benchmark to regenerate the synthetic benchmark report
• make eval-hc3 to regenerate the HC3 evaluation report
• upload of docs/BENCHMARK.md and docs/HC3_EVALUATION.md as workflow artifacts

See CONTRIBUTING.md.

Good contributions usually improve one of these:

• clearer evidence signals
• safer wording and UX
• multilingual handling that stays explainable
• reproducible evaluation coverage
• agent integration ergonomics

🔌 Visit lynote.ai/ai-detector-api to get your AI Detector API key.

MIT