Xberg
The fast, precise document-intelligence engine — for every language.
Point Xberg at anything — a PDF, a scanned image, a spreadsheet, an audio file, a URL, a whole archive, or a source tree — and get back clean text, tables, metadata, and structured data. One engine handles format detection, reading, OCR, and extraction, so you never stitch a pipeline together from a dozen libraries.
107 formats · 141 file extensions · 371 code languages · 15 language bindings · 6 output formats · OCR · transcription · embeddings
The fastest, most precise open-source document and PDF-to-Markdown engine — see the benchmarks.
Install · What you get · Capabilities · CLI · Docs
Xberg is the next iteration of Kreuzberg. Same document-intelligence engine, rebuilt and rebranded under a fresh v1 line.
What you get
Point Xberg at anything — a PDF, a spreadsheet, a scanned image, an audio file, a URL, an archive, a source tree — and get back clean, structured content you can use right away. One core does the format detection, reading, and extraction, so you don't assemble a pipeline yourself. Call it from Rust, Python, Node.js, Go, Java, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WASM, Kotlin, or C FFI, and run it as a library, CLI tool, REST API, or MCP server.
| Capability | What you get |
|---|---|
| 107 document formats | PDFs, Office, images, HTML, email, e-books, scientific publications, and structured data across 141 file extensions, with intelligent MIME detection and bounded extraction controls. |
| URLs & the web | Point Xberg at an http(s) URL — it fetches and extracts a single document, or crawls and follows links (Auto / Document / Crawl modes via the crawlberg engine). Requires the url-ingestion feature. |
| Audio & video transcription | Speech-to-text from MP3, M4A, WAV, WebM, and MP4 tracks via Whisper ONNX (tiny → large-v3). Requires the transcription feature. |
| Archives, traversed | List and recursively extract nested .zip, .tar, .gz, .7z — documents inside documents — guarded by zip-bomb, compression-ratio, and nesting-depth limits. |
| OCR on demand | Tesseract, PaddleOCR, Candle, or VLM backends — fallback chains, confidence scores, language auto-detection, extensible via plugins. |
| Layout & tables | ML layout models (PP-DocLayout-V3, RT-DETR) and table structure (TATR, SLANet) reconstruct reading order and cell grids for clean Markdown. |
| Code intelligence | Functions, classes, imports, symbols, docstrings from 371 programming languages. Syntax-aware chunking for RAG pipelines. |
| Embeddings & search | Local (ONNX) or provider-hosted embeddings (165 providers via liter-llm), sparse and late-interaction, cross-encoder reranking. |
| Enrichment | NER, keyword extraction (YAKE/RAKE), summarization, translation, redaction, page classification, QR detection, language detection, token reduction (TOON). |
| Structured extraction | Schema-driven JSON straight from any document via local (Ollama, LM Studio, vLLM) or hosted LLMs — no prompt engineering. |
| 6 output formats | Plain text, Markdown, Djot, HTML, JSON tree, or Docling DocTags, plus registered custom renderers. |
| Runs anywhere | Library, CLI (14 commands), REST API (xberg serve), MCP server, Docker, Helm — CPU by default, no GPU required. Content-hash caching, parallel batch, per-file timeouts. |
Capabilities marked requires a feature are Cargo feature flags on the core crate (
url-ingestion,transcription,reranker, layout/ORT). Prebuilt language packages and the Docker image bundle the common set; a from-source build enables only what you select.
Installation
Language Packages
Go
go get github.com/xberg-io/xberg/packages/go@latest
⚠️ The repository root is not a Go module —
go get github.com/xberg-io/xbergwill fail. Always target the/packages/gosubdirectory as shown above.
See Go README for full documentation.
Java
Available on Maven Central as io.xberg:xberg. See Java README for the dependency snippet.
Elixir
Add {:xberg, "~> 1.0"} to your mix.exs dependencies. See Elixir README for full documentation.
Kotlin (Android)
Available on Maven Central as io.xberg:xberg-android. See Kotlin README for the dependency snippet.
Swift
Add via Swift Package Manager. See Swift README for full documentation.
Zig
Add via zig fetch. See Zig README for full documentation.
C/C++ (FFI)
Build from source as part of this workspace. See C (FFI) README for full documentation.
CLI & Deployment
CLI Tool
brew install xberg-io/tap/xberg
Windows users can install the same binary through Scoop:
scoop bucket add xberg https://github.com/xberg-io/scoop-bucket scoop install xberg
14 commands: extract, batch, detect, formats, version, cache, tree-sitter, doctor, serve, mcp, api, embed, chunk, and completions.
See CLI usage guide for detailed documentation.
Docker
docker pull ghcr.io/xberg-io/xberg:latest
Run in API, CLI, or MCP modes. See Docker guide for examples.
REST API Server
xberg serve --host 0.0.0.0 --port 8000
One POST endpoint handles all formats. Returns JSON or Markdown. Stream large files. See API server guide.
MCP Server
xberg mcp --transport stdio
9 tools (extract, extract_batch, detect_mime_type, cache_stats, list_formats, cache_clear, get_version, cache_manifest, cache_warm). 3 prompts (extract_document, extract_with_ocr, semantic_search). 4 resources (formats, models, OCR languages, embedding presets).
Add to Claude Desktop or Cursor:
{
"mcpServers": {
"xberg": { "command": "xberg", "args": ["mcp"] }
}
}AI Coding Assistants
Install the Xberg plugin from xberg-io/xberg. Ships extraction APIs, OCR backends, configuration, and language conventions.
Claude Code
/plugin marketplace add xberg-io/xberg
/plugin install xberg@xberg
Codex CLI
/plugins add https://github.com/xberg-io/xberg
Search for xberg and select Install Plugin.
Cursor
Settings → Plugins → Add from URL → https://github.com/xberg-io/xberg, then select xberg.
Gemini CLI
gemini extensions install https://github.com/xberg-io/xberg
Factory Droid
droid plugin marketplace add https://github.com/xberg-io/xberg
droid plugin install xberg@xberg
GitHub Copilot CLI
copilot plugin marketplace add https://github.com/xberg-io/xberg
copilot plugin install xberg@xberg
Pi
Install from a local checkout now:
pi install ./plugin
After the next plugin release, install the same package used by OpenCode:
pi install npm:@xberg-io/opencode-xberg
Pi loads the bundled skills and prompt templates. Use /xberg-extract and /xberg-ocr
to start a task. Install the CLI separately as
shown in the plugin README.
This package does not register MCP servers, hooks, or executable extensions in Pi.
opencode
Add to opencode.json:
{
"$schema": "https://opencode.ai/config.json",
"plugin": ["@xberg-io/opencode-xberg"]
}Quick Start
Extract text from a document:
use xberg::{extract, ExtractInput, ExtractionConfig}; #[tokio::main] async fn main() -> xberg::Result<()> { let config = ExtractionConfig::default(); let output = extract( ExtractInput::from_uri("document.pdf"), &config ).await?; println!("{}", output.results[0].content); Ok(()) }
Common use cases — see Quick start guide for language-specific examples, OCR, batch processing, and API configuration.
Capabilities
Full feature list
Supported File Formats (107 formats · 141 file extensions · 56 MIME aliases)
107 formats across 140 unique file extensions, with 56 compatibility MIME aliases, intelligent format detection, and comprehensive metadata extraction.
Office Documents
| Category | Formats | Capabilities |
|---|---|---|
| Word Processing | .docx, .docm, .doc, .dotx, .dotm, .dot, .odt, .pages, .wpd, .wp, .wp5, .wp6 |
Full text, tables, images, metadata, styles |
| Spreadsheets | .xlsx, .xlsm, .xlsb, .xls, .xla, .xlam, .xltm, .xltx, .xlt, .ods, .numbers |
Sheet data, formulas, cell metadata, charts |
| Presentations | .pptx, .pptm, .ppt, .pps, .ppsx, .potx, .potm, .pot, .odp, .key |
Slides, speaker notes, images, metadata |
.pdf |
Text, tables, images, metadata, OCR support | |
| eBooks | .epub, .fb2 |
Chapters, metadata, embedded resources |
| Database | .dbf, .sqlite, .sqlite3, .db, .gpkg, .gpkx |
Bounded table extraction, schema metadata, GeoPackage detection |
| Hangul | .hwp, .hwpx |
Korean document format, text extraction |
Images (OCR-Enabled)
| Category | Formats | Features |
|---|---|---|
| Raster | .png, .jpg, .jpeg, .gif, .webp, .bmp, .tiff, .tif |
OCR, table detection, EXIF metadata, dimensions, color space |
| Advanced | .jp2, .jpg2, .j2c, .j2k, .jpc, .jbig2, .jb2, .pnm, .pbm, .pgm, .ppm |
OCR via pure-Rust JPEG2000 decoder, JBIG2 support, table detection |
| HEIC family | .heic, .heics, .heif, .heifs, .hif, .avif, .avcs |
EXIF metadata, optional pixel decoding |
| Vector | .svg |
DOM parsing, embedded text, graphics metadata |
Audio & Video
| Category | Formats | Features |
|---|---|---|
| Audio | .mp3, .mpga, .m4a, .wav, .webm |
Whisper transcription |
| MP4 audio track | .mp4, .mpg4, .mp4v, .m4v |
Audio-track transcription only |
| MPEG audio track | .mpeg, .mpg, .mpe, .m1v, .m2v |
Audio-track transcription only |
| WebM audio track | .webm |
Audio-track transcription only |
Web & Data
| Category | Formats | Features |
|---|---|---|
| Markup | .html, .htm, .xhtml, .xht, .xml, .kml, .svg |
DOM parsing, metadata (Open Graph, Twitter Card), link extraction |
| Structured Data | .json, .geojson, .jsonl, .ndjson, .yaml, .yml, .toml, .csv, .tsv |
Schema detection, nested structures, validation |
| Text & Markdown | .txt, .adoc, .asciidoc, .vtt, .md, .markdown, .commonmark, .qmd, .rmd, .djot, .dj, .mdx, .doctags, .rst, .org, .rtf |
AsciiDoc, CommonMark, MyST Markdown, Quarto, R Markdown, Djot, MDX, DocTags, reStructuredText, Org Mode |
Email & Archives
| Category | Formats | Features |
|---|---|---|
.eml, .msg, .pst |
Headers, body (HTML/plain), attachments, threading | |
| Archives | .zip, .tar, .tgz, .gz, .7z |
File listing, nested archives, metadata, recursive extraction |
Academic & Scientific
| Category | Formats | Features |
|---|---|---|
| Citations | .bib, .ris, .nbib, .enw |
Structured parsing: RIS, PubMed/MEDLINE, EndNote XML, BibTeX/BibLaTeX |
| Scientific | .tex, .latex, .typ, .typst, .jats, .nxml |
LaTeX, Typst, PubMed JATS |
| Text notebooks | .ipynb, .md, .py, .R, .jl |
Jupyter, MyST-NB, Jupytext percent/light, saved outputs, cell visibility tags |
| Publishing | .fb2, .docbook, .dbk, .docbook4, .docbook5, .opml |
FictionBook, DocBook XML, OPML outlines |
Code Intelligence (371 Languages)
Extract structure from 371 programming languages via tree-sitter:
| Feature | Description |
|---|---|
| Structure Extraction | Functions, classes, methods, structs, interfaces, enums |
| Import/Export Analysis | Module dependencies, re-exports, wildcard imports |
| Symbol Extraction | Variables, constants, type aliases, properties |
| Docstring Parsing | Google, NumPy, Sphinx, JSDoc, RustDoc, and 10+ formats |
| Syntax-Aware Chunking | Split code by semantic boundaries for RAG pipelines |
| Diagnostics | Parse errors with line/column positions |
Powered by tree-sitter-language-pack.
Output Formats (6)
| Format | Use case | Example |
|---|---|---|
| Plain | Raw text, no markup | "Chapter 1\nIntroduction" |
| Markdown | Readable, structured, RAG-friendly | "# Chapter 1\n## Introduction" |
| Djot | Modern lightweight markup | Similar to Markdown but stricter |
| HTML | Styled, browser-ready | <h1>Chapter 1</h1> |
| JSON | Machine-readable tree structure | Hierarchical sections with heading levels |
| DocTags | Docling-compatible tag stream for document elements and tables | <text>Chapter 1</text> |
Deployment Modes
| Mode | Command | Transport | Use case |
|---|---|---|---|
| Library | xberg::extract() |
Async functions | Embed in your application |
| CLI | xberg extract document.pdf |
14 commands | Scripts, batch jobs, CI/CD |
| REST API | xberg serve |
HTTP POST | Microservice, serverless deployment |
| MCP Server | xberg mcp |
stdio or HTTP | Claude, Cursor, IDE agents |
| Docker | docker run ghcr.io/xberg-io/xberg |
All modes | Container deployment |
OCR Backends
- Tesseract — Native C FFI (Linux/macOS/Windows) and WASM (browser)
- PaddleOCR — ONNX Runtime, mobile-optimized models
- Candle — Pure Rust, CPU-only, lightweight
- VLM — GPT-4 Vision, Claude Vision, Gemini Vision, or 165 providers via liter-llm
Fallback chains. Extensible via plugin system.
Embeddings
Local (ONNX Runtime):
- Preset models: fast, balanced (default), quality, multilingual
- Dimensions: 384, 768, 1024
Provider-hosted:
- OpenAI, Anthropic, Google, Hugging Face, Mistral, Cohere, and 165 providers total
- Via liter-llm integration
Reranking:
- Local ONNX rerankers (cross-encoder models)
- Provider-hosted: Cohere Rerank, others
Structured LLM Extraction
Local engines: Ollama, LM Studio, vLLM
Remote: OpenAI, Anthropic, Google, Mistral, Cohere, and 165 providers via liter-llm
Schema validation. Temperature, top-p, frequency penalty tuning.
Enrichment
- NER — GLiNER or LLM-based entity recognition
- Redaction — Mask PII (phone, email, SSN, credit card, addresses)
- Summarization — Document and section summaries via LLM
- Translation — Multi-language via LLM
- Page Classification — Tag document pages (cover, toc, content, etc.)
- QR Code Detection — Extract and decode QR codes from images
- Keyword Extraction — YAKE or RAKE algorithms
- Language Detection — Detect document language
- Layout Detection — RT-DETR + TATR models for document structure
- Table Extraction — Cell-level structure and content
- Token Reduction — TOON wire format (~30–50% fewer tokens than JSON)
CLI Reference
All 14 commands
| Command | Subcommands | Purpose |
|---|---|---|
extract |
— | Extract text from a single document (path, URL, or stdin) |
batch |
— | Extract from multiple documents in parallel |
detect |
— | Identify MIME type of a file |
formats |
— | List all supported formats and MIME types |
version |
— | Show Xberg version |
cache |
stats, clear, manifest, warm |
Manage extraction cache and models |
tree-sitter |
download, list, cache-dir, clean |
Manage code-intelligence grammars |
doctor |
— | Diagnose the local installation and runtime dependencies |
serve |
— | Start REST API server (default: http://127.0.0.1:8000) |
mcp |
— | Start MCP server (stdio or HTTP transport) |
api |
schema |
Output OpenAPI 3.1 specification |
embed |
— | Generate embeddings for text (local or provider-hosted) |
chunk |
— | Split text into chunks (text, markdown, YAML, or semantic) |
completions |
— | Generate shell completion scripts |
Run xberg --help or xberg <command> --help for detailed options.
Documentation
Full guides, API references for every binding, format reference, and configuration docs live at xberg.io.
- Getting Started
- Quick Start
- Guides
- API Reference (Rust core) — every binding has its own page under
/reference/ - Format Reference
- Live Demo (browser, WASM)
Built with Xberg
Projects that declare Xberg as a dependency. Xberg was previously published as kreuzberg, and most of these projects declare the package under that name.
| Project | What it is | Stars |
|---|---|---|
| basemind | AI context and content layer for coding agents over one MCP server: code map, document RAG, shared memory and web crawl | |
| delulu | A suite of MCP servers and CLI tools that give your LLM better search and fewer hallucinations | |
| docs-mcp-server | Grounded documentation MCP server, an open-source alternative to Context7, Nia and Ref.Tools | |
| erato | The open-source AI platform | |
| fastmail-cli | CLI and MCP server for Fastmail: email, contacts, masked email, attachments and text extraction | |
| ghfdb-portal | Web portal for the Global Heat Flow Database | |
| hawki-toolkit-file-converter | Prepares and converts PDF files for the HAWKI toolkit | |
| haystack-core-integrations | Integrations that extend Haystack with extra components and document stores | |
| kreuzakt | A search engine for humans and computers, aimed at your most boring documents | |
| lilbee | The whole local AI stack in one executable, with conversational search and cited answers over your files, code and the web | |
| llm-workflow-engine | Power CLI and workflow manager for LLMs | |
| MANSPIDER | Spiders entire networks for files sitting on SMB shares, searching filenames or contents with regex | |
| otoroshi-llm-extension | Connect, secure and manage LLM models behind one OpenAI-compatible API | |
| sift-kg | Turns a collection of documents into a knowledge graph, extracting entities and relationships with an LLM | |
| sirchmunk | Turns raw data into a self-evolving, real-time search and intelligence layer | |
| support-chatbot | Level-1 support chatbot for the Netherlands Red Cross 510 team |
Using Xberg in your project? Open a PR adding it to this list.
Contributing
Contributions are welcome! See CONTRIBUTING.md for guidelines.
Join our Discord community for questions and discussion.
Part of Xberg.io
- Xberg — the open-source content-intelligence engine: text, tables, and metadata from 107 formats (141 file extensions), with OCR, transcription, and code intelligence. MIT.
- Xberg Pro — a complete self-hosted content-intelligence backend in a single container. Commercial.
- Xberg Enterprise — the distributed, governed content-intelligence platform, scaled on Kubernetes with team governance and support. Commercial.
- crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
- html-to-markdown — fast, lossless HTML→Markdown engine.
- liter-llm — universal LLM API client with native bindings for 14 languages and 165 providers.
- tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
- alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.
License
MIT License (MIT) — see LICENSE for details.