PocketGPT on ESP32-S3
A language model trained from scratch and running offline on an ESP32-S3. Use a touchscreen keyboard on XIAO or chat over USB on DevKit.
Watch the project video · Hardware on Amazon
Run PocketGPT
The trained weights and tokenizer are available. No training or API key is needed. Download the source, build the firmware and install the matching model package using the guide for your board:
| Your hardware | Start here |
|---|---|
| XIAO ESP32-S3: 8 MB Flash / 8 MB PSRAM, Seeed Round Display and microSD | Touchscreen setup |
| ESP32-S3 DevKit: 32 MB octal Flash / 16 MB PSRAM | USB chat setup |
Start with the shared preparation steps at the top of the installation guide. Model packages are in Releases; firmware is built from this repository. The two boards use different packages.
What can PocketGPT actually do?
This is a personal experiment with limited capabilities. It can produce short greetings and basic conversational replies, but answers can be wrong, repetitive or unrelated. Even basic arithmetic is unreliable.
These are recorded hardware outputs:
| Prompt | Response | Board |
|---|---|---|
hello |
Hello. |
XIAO |
how are you? |
I'm doing okay. |
XIAO |
say something nice |
You're doing better than you think. |
DevKit |
The XIAO tests took about 6.4 seconds for hello and 11.8 seconds for how are you?. These examples do not guarantee the same quality for other questions. XIAO recordings · DevKit recording
Model specifications
| Characteristic | Value |
|---|---|
| Independent trained parameters | 30,460,864 |
| Architecture | Decoder-only Transformer; 12 layers; hidden size 448 |
| Attention | 8 query heads, 2 key/value heads |
| Tokenizer | SentencePiece; 6,144 tokens |
| Runtime limits | 64-token context; up to 24 generated tokens per response |
| Recorded generation speed | DevKit: about 1.41 tokens/s; XIAO: about 0.742 tokens/s |
| Deployment files | DevKit: 20,241,616-byte deduplicated pack. XIAO: 21,717,088 bytes of model weights across Flash, PSRAM-loaded data and microSD; cache and tokenizer files are separate. See the exact XIAO layout. |
Speeds are individual measurements, not guarantees. The packages use different mixed-precision layouts. See model and quantization details.
Training
The original training code and settings are included. Base V2 names the second base-model version, trained from random initialization on about 300 million token positions before later refinement. Exact historical retraining requires additional data and records described in that guide.
Optional: Gemini or Ollama
The network interface uses the same XIAO/display hardware, but runs a different firmware. Answers come from Gemini's servers or Ollama on your PC. This option does not use PocketGPT's weights.
Serial testing
After installing PocketGPT on a DevKit, follow the USB test instructions to send a prompt and save the result. The test checks for a nonempty reply, a positive token count and no reported inference errors. Passing confirms generation completed, not that the answer is factually correct.