Utopia
The enterprise world model built by DeepLethe. It is the first open substrate for knowledge engineering that learns passively and governs itself. Where a knowledge graph or a vector store works to hold present knowledge, Utopia puts time awareness and ontology in the base layer: the knowledge system evolves as material arrives, and conflict detection, reasoning and decision making all run against that ontology. It deploys offline, so a company can stand up a knowledge foundation, a decision core its agents can trust, and a compliance audit trail on hardware it controls.
Please note: we would rather this project were not framed as an open-source take on Palantir. It is a different route to enterprise intelligence, built bottom up from knowledge governance to trustworthy decisions and simulation.
Utopia.Bitemporal.graph.and.Derivation.mp4
Philosophy
We gave it a somewhat romantic name, Utopia. Ptolemy's geocentric model was taken for truth for a very long time, then falsified step by step by Copernicus, Kepler, Galileo and Newton. Looking back, what we keep is not only that heliocentrism turned out to be right; it is how that history unfolded.
Where existing vector stores and knowledge graphs work to get present knowledge right, one of Utopia's founding aims is to record the whole course of changing understanding. Engineered, that becomes a bitemporal knowledge graph. When a decision is reviewed later, the system can produce the full course it took and the grounds it rested on. To make this hold up in practice we have iterated at length against public corpora spanning enterprise records, education, finance, law and research. Temporality is only one facet; for how knowledge is taken in, how the future is reasoned about, and how logic bounds action, see utopia.bi/philosophy.
Use cases
What Utopia is for, in the shape of the question someone brings. Each is available today unless it says otherwise; each needs a chat model configured, and each leaves the doubtful cases to a person in Review.
| Who and what | Input | Question, and what comes back | What the base adds over search or RAG |
|---|---|---|---|
| A contract chain. Legal or procurement, a master agreement amended several times. | The agreement and its amendments, as PDF or Word. | Which payment terms applied on 15 March 2024, and who were the parties? The terms in force on that date, the amendment they came from, the sentence. | Each fact carries when it held and where it was said; a later amendment closes the earlier term instead of overwriting it. The contracts corpus is this scenario, with the questions it has to answer. |
| Who held what, when. HR, finance or research records that change: handovers, corrections, restatements. | Announcements, minutes, filings, dated as they were written. | Who led project X in Q2 2023, and what did we believe about it at the time? The holder for that period, and separately what the base knew then. | Two clocks: when something was true, and when the base learned it. A correction links to what it replaced. The temporal QA bench measures this. |
| An agent that reads the base. Engineering or operations teams with Claude Desktop, Cursor or their own agent. | An existing base; the agent connects over MCP with a token scoped to it. | Facts about this entity as of a date; the path between two things; what changed last week. Structured answers with stable identifiers and evidence; anything the agent wants to record goes to a person first. | Reads are typed and dated, not passages; writes are proposals. The MCP guide lists the tools. |
| Observations from a system. Robotics, monitoring, security tooling: a system that already knows what it saw and when. | One push per observation, in the extraction contract itself, with the observation's time. New in rc7. | Where was this object at 08:05, and where is it now? The value that held at that moment and the observation it rests on. | No model between the system and the graph; a functional attribute closes its earlier value when a later observation arrives. The walkthrough runs this end to end with curl. |
The walkthrough is the shortest way to see the whole path: sample data, ingestion, a question, the answer with its evidence, and what the base does not do.
Features
One Rust binary and one Postgres. Full-text search is embedded in the binary, vectors go in pgvector, and the job queue is a table: nothing else to run.
| A complete application | A system console, a graph browser and an ontology workbench in one web UI. A product, not a library: install it and it works. |
| Knowledge ingest | Upload PDF, DOCX, PPTX, XLSX, XLS, ODS, CSV, TSV, Markdown, HTML or plain text, with legacy encodings detected on the way in. Scans and images are read by a MinerU service or a Volcengine Ark vision model, recordings by a transcription model that labels speakers. Web pages, RSS, GitHub, Jira, Notion, WebDAV and S3-compatible buckets sync on a schedule; everything else comes in through the API. |
| Search and chat | Full-text on Tantivy, vectors on pgvector, fused with RRF. Answers stream with inline citations that open the passage they came from. Any OpenAI-compatible endpoint works (DeepSeek, Qwen, GLM, Ollama, vLLM), so the whole system can run air-gapped. |
| Agent harness and agentic RAG | The whole system can be driven through conversation. The built-in agent searches documents, walks the graph (an entity's facts as of any date, or what changed in a period) and queries a mounted database. The same read-only tools are exposed over MCP. |
| Agents over MCP | An MCP server for every knowledge base, so Claude Desktop, Cursor, Workbuddy and other agent frameworks can connect, with fine-grained permissions. |
| Ontology and cold start | A new knowledge base has no vocabulary of its own; it starts from the packs you pick at creation. Five ship inside the binary: schema.org, W3C Org, PROV-O, FOAF and IOF Core (ask for your industry). Terms outside the packs are counted as they appear; confirm the common ones and they join the ontology. |
| Bitemporal graph | Extraction turns documents into entities and facts, following an ontology you can edit. Every fact carries when it held and where it came from. Correcting a fact closes the old version and links the new one to it rather than overwriting, so the graph keeps two timelines: when something was true in the world, and when the system came to believe it. Edges are reified, so an edge carries attributes of its own. |
| Entity resolution and review | Duplicates are resolved in three stages: exact name or alias, embedding similarity, then a model's call on the doubtful pairs. Every merge can be undone. Uncertain cases go to a review queue: low-confidence extractions, suspected duplicates and cardinality conflicts. |
| Agent adjudication | The agent adjudicates on its own, from common sense and the business documents. Low-confidence calls go to the human review queue, and the decisions people make there are recorded and used to tune the agent, closing the loop. |
| Reasoning and derivation | Ontology axioms compile into rules: transitivity, symmetry, inverses and relation hierarchy derive new facts by forward chaining. Derivation is off by default, since a wrong axiom derives wrong facts. A derived fact is marked as such on the graph, carries validity and confidence like any other, and shows what it was derived from. When it contradicts an asserted fact, the asserted one stands. |
| Conflict detection | Three kinds of conflict, three sets of choices. A new fact that clashes with an older one: close the old, keep both, or reject the new. Data that breaks an axiom (self-loop, asymmetry, transitive cycle, cardinality): retract the fact, relax the axiom, or accept both. The ontology itself is checked first, because violations of a self-contradictory ontology are noise. |
| Ontology-driven querying | Mount a database on a base (Postgres, MySQL and the engines that speak its protocol, ClickHouse, Trino for Iceberg / Delta Lake / Hive, Databricks, Snowflake) and chat can query it alongside the documents. The agent proposes how its tables map onto the ontology, and you confirm. The method behind it, Ontology2SQL, is state of the art on BIRD Mini-Dev for SQLite and PostgreSQL (submission). |
| Multi-user and permissions | Each knowledge base has its own members and roles: owner, admin, editor and viewer. Open bases are readable by everyone in the deployment, restricted ones only by invitation. The first account registered becomes the system administrator. |
| Decision ledger | Confirming or rejecting a fact, merging or reverting an entity, rebuilding the graph: each leaves a record of who, when, and what the object looked like at the time. The ledger is append-only, and a record outlives its object, even the base it belonged to. |
| Decision intelligence (in development) | Record a decision, replay both what was understood and the course it took, and reason over overlaid scenarios. |
Quick start
Requirements: Docker (local development also needs Rust 1.85+, Node 20+, pnpm).
The prebuilt image is published for linux/amd64 and linux/arm64, so it runs natively on x86-64 hosts as well as Apple Silicon and AWS Graviton. Tags up to 0.1.0-rc7 are amd64 only.
Start from the prebuilt image:
git clone https://github.com/deeplethe/utopia.git
cd utopia
docker compose --profile app up -dOpen http://localhost:1516 and register. The first account automatically becomes the administrator, and a public knowledge base readable by everyone is created at the same time. Before extracting business documents, configure the model endpoints (chat and embedding) under Administration → Models.
The database password (UTOPIA_DB_PASSWORD in .env, default utopia) is applied when the data volume is first initialised. To change it on a running deployment, change it in the database as well — docker compose exec db psql -U utopia -c "ALTER USER utopia PASSWORD '<new password>'" — or start over with docker compose --profile app down -v, which deletes all data.
Or build from source:
docker compose -f docker-compose.yml -f docker-compose.build.yml --profile app up -d --build
Local development
# 1. Postgres with pgvector docker compose up -d db # 2. Backend on :1516, runs migrations on startup cargo run -p utopia-server # 3. Frontend on :5173, proxying /api to the backend cd web && pnpm install && pnpm dev
Roadmap
- Decision reasoning: constraint computation, and replaying a decision after the fact
- Business rules: rules written by people over an entity's attribute facts, a threshold or a category set, that classify it as a derived fact with the rule and the premises as its explanation (#277)
- Execution gate: checking an agent's calls against ontology rules and symbolic logic
- MaxCompute: mapping exploration and Ontology2SQL over Alibaba Cloud MaxCompute (Iceberg / Delta Lake via Trino, Databricks and Snowflake are in, awaiting a run against a real cluster)
- More sources: a Feishu connector
- Agent memory over MCP: episode writes, the retrieve endpoint, and the MCP server
- Enterprise: OIDC SSO, backup and restore commands, benchmarks at 100k documents
Status
Utopia is still at v0.1. The database schema evolves between versions and migrations only roll forward, with no rollback. Pin a specific version with UTOPIA_IMAGE in production, and back up the database along with the data directory before upgrading.
Please read SECURITY.md before exposing it to the public internet.
Star History
Community
- 💬 Discussions: discuss the project, share your experience, and leave feedback
- 🐛 Issues: report bugs, ask design questions, and submit feature requests
- 🤝 Contributing: development setup, pre-push checks, and DCO sign-off
- 🔌 Ontology2SQL: the ontology-driven text-to-SQL method referenced above