nobodywho-ooo/nobodywho

On-device AI for any device.
NobodyWho is an inference engine that lets you run LLMs locally and efficiently.

• Run locally, offline — no API keys needed or hidden fees
• Run any chat LLM — Gemma, Qwen, Mistral and more
• Fast, type-safe tool calling — automatically generates structured grammars from your function signatures, no schema writing needed
• Multimodal input — provide image and audio information to your LLM
• Text-to-speech — synthesize local WAV audio with Kokoro, Pocket TTS and Supertonic backends
• Speech-to-text — transcribe audio into text with Whisper
• Voice Activity Detection — know when to stop listening and start transcribing with Silero
• Model downloading — load models directly from Hugging Face or any URL

flowchart TD
    FL["Flutter"]:::lang
    PY["Python"]:::lang
    GO["Godot"]:::lang
    K["Kotlin"]:::lang
    S["Swift"]:::lang
    RN["React Native"]:::lang

    FRB["flutter_rust_bridge"]:::glue
    P3["PyO3"]:::glue
    GX["gdext"]:::glue
    U["UniFFI"]:::glue

    FL --> FRB
    PY --> P3
    GO --> GX
    K --> U
    S --> U
    RN --> U

    CORE["NobodyWho · Rust<br/>chat · templates · grammars · sampling · context shifting"]:::core

    FRB --> CORE
    P3 --> CORE
    GX --> CORE
    U --> CORE

    CORE --> LCPP["llama.cpp<br/>text · vision · embeddings · reranking"]:::engine
    CORE --> ORT["ONNX Runtime<br/>speech-to-text · text-to-speech · VAD"]:::engine

    LCPP --> HW["Hardware<br/>Vulkan · Metal · CUDA · GPU · CPU"]:::hw
    ORT --> HW

    classDef lang fill:#e8eefc,stroke:#5b7bd5,color:#11204a
    classDef glue fill:#f3f0fb,stroke:#8b7bd5,color:#2a1f4a
    classDef core fill:#eaf6ee,stroke:#4fa46a,color:#0f3b1f
    classDef engine fill:#fdf0e3,stroke:#d58f3b,color:#4a2d0b
    classDef hw fill:#f0eeea,stroke:#a09a90,color:#3a352e

• GPU-accelerated inference via Vulkan or Metal — runs fast on any OS
• Compatible with thousands of pre-trained LLMs — use any LLM in the GGUF format
• Powered by the wonderful llama.cpp

You can test our inference engine on iOS, Android, Vision Pro and Apple Watch.

Desktop means Linux, macOS and Windows throughout. Three gaps worth knowing before you start:

• Godot has no iOS export. Use the Flutter, React Native or Swift binding on iOS.
• Windows ARM64 is not supported yet.
• There is no web export. It is tracked in issue #111.

⭐ Useful to you? Star the repo, it's the easiest way to say thanks.

Inference uses the GPU where available and CPU where not. The real constraint is memory.

• Hardware — any 64-bit Linux, macOS or Windows machine. Windows is x86_64 only, there are no ARM64 Windows builds. macOS accelerates through Metal out of the box, Linux and Windows need a GPU driver with Vulkan support and fall back to CPU without one.
• Memory — roughly 1.5× the model file in free RAM, or 2× on a machine that's already busy. 8 GB is a comfortable floor for models up to ~2 GB; 16 GB or more above that.
• Discrete GPUs — NobodyWho offloads as many layers as fit in free VRAM and runs the rest on the CPU, so a model larger than your VRAM still works, just slower. If barely any of it fits, it skips the GPU and stays on CPU. It uses one GPU, the card with the most free memory.

• iOS — iPhone 11 or newer, 4 GB RAM or more.
• Android — Snapdragon 855 / Adreno 640 / 6 GB RAM or better.
• Rule of thumb — the device needs roughly twice the model file size in available RAM. iOS reserves around 2 GB, Android 2 to 4 GB depending on vendor. Models under 1 GB run smoothly on any phone.

Any model in the GGUF format works. Pass a hf:owner/repo:QUANT reference, an HTTPS URL, or a local path anywhere a model path is expected; remote models are downloaded and cached on first use. Pass "auto" to fit one to available memory.

Start with Qwen3 0.6B: at ~330 MB it is small enough for any phone and good enough to tell whether the integration works.

pip install nobodywho

from nobodywho import Chat

chat = Chat('hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M')
response = chat.ask('What is the capital of Denmark?')
print(response.completed()) # The capital of Denmark is Copenhagen.

// Android
implementation("ai.nobodywho:nobodywho-android:<version>")

// Desktop JVM (Linux, macOS, Windows)
implementation("ai.nobodywho:nobodywho:<version>")

import ai.nobodywho.Chat

val chat = Chat.fromPath(
    modelPath = "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M"
)

val response = chat.ask("What is the capital of Denmark?").completed()
println(response) // The capital of Denmark is Copenhagen.

https://github.com/nobodywho-ooo/nobodywho-swift.git

import NobodyWho

let chat = try await Chat.fromPath(
    modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M"
)

let response = try await chat.ask("What is the capital of Denmark?").completed()
print(response) // The capital of Denmark is Copenhagen.

# React Native
npm install react-native-nobodywho

# Expo
npx expo install react-native-nobodywho

import { Chat } from "react-native-nobodywho";

const chat = await Chat.fromPath({
  modelPath: "hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M",
});

const msg = await chat.ask("What is the capital of Denmark?").completed();
console.log(msg); // The capital of Denmark is Copenhagen.

flutter pub add nobodywho

import 'package:nobodywho/nobodywho.dart' as nobodywho;

void main() async {
  await nobodywho.NobodyWho.init();

  final chat = await nobodywho.Chat.fromPath(
    modelPath: 'hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M',
  );

  final msg = await chat.ask('What is the capital of Denmark?').completed();
  print(msg); // The capital of Denmark is Copenhagen.
}

uvx --from 'git+https://github.com/nobodywho-ooo/nobodywho.git#subdirectory=nobodywho/server' nobodywho-server --model hf:NobodyWho/Qwen_Qwen3-0.6B-GGUF:Q4_K_M --name qwen

The documentation has everything you might want to know: https://docs.nobodywho.ooo/

Working with a coding agent? Point it at llms.txt or llms-full.txt, or install the NobodyWho skill so your agent can look up the current APIs and documentation:

npx skills add https://github.com/nobodywho-ooo/nobodywho --skill nobodywho

• Discord & Matrix — ask us anything
• Issues and Discussions — bugs and feature requests
• CONTRIBUTING.md — set up the repo and send a PR
• CHANGELOG.md · SECURITY.md · CODE_OF_CONDUCT.md

⭐ Star the repo if NobodyWho is useful to you. It is how people find us.

NobodyWho is licensed under the EUPL-1.2. You may use it in proprietary and commercial projects, free of charge. There has been some confusion about this, so to be precise:

Linking two programs or linking an existing software with your own work does not – at least under European law – produce a derivative or extend the coverage of the linked software licence to your own work. [1]

If you distribute modified versions of the code in this repo, you must open source those changes.

Feel free to make proprietary projects using NobodyWho, but don't make a proprietary fork of NobodyWho.