Dicklesworthstone/frankensonos

FrankenSonos

Your Sonos, your way. Command your speakers from anywhere in the world, as long as you're on your Tailscale tailnet. A memory-safe Rust controller and daemon for the Sonos speakers you already own: one binary that runs the whole house from the terminal, an HTTP API, or any AI agent that speaks MCP.

Status: pre-release (before 0.1.0). Everything below is implemented, and every feature is tested end to end against fsonos-sim, a virtual Sonos house (an S1 and an S2 household speaking SOAP and GENA on loopback, checked against scrubbed captures of real players). Live tests against real players exist but are opt-in; there is no real-hardware CI yet, and interfaces may still change before 0.1.0.

Sonos hardware is excellent and lasts for years; the software around it is where it falls short. The app loses speakers, grouping fights you, automation is out of reach, the legacy S1 line is frozen while S2 moves on without it, and the Spotify Web API cannot start playback on a Sonos at all.

FrankenSonos talks to your speakers directly, over the local protocols they already speak (SSDP discovery, UPnP/SOAP on port 1400, GENA events), and turns the whole house, both generations at once, into something you and your agents can drive:

• From anywhere on your tailnet. fsonos serve serves its HTTP API on loopback and your Tailscale addresses (MCP on loopback), and fsonos tailscale setup puts HTTPS in front of both with Tailscale Serve. Your phone in another city, an agent on a cloud VM, a laptop at work: if it is on your tailnet, it runs your house (the CLI there too: FSONOS_DAEMON_URL=https://<mac>.<tailnet>.ts.net fsonos pause Kitchen). Nothing is exposed to the public internet, Funnel is never used, and the speakers never leave your LAN.
• Built for agents, with guardrails. Every surface (CLI, HTTP, MCP) goes through one control path: the house policy (per-room volume caps, quiet hours, per-client tool allowlists), an action log of who did what, and undo. The volume caps and quiet hours bind the agent surfaces; the local CLI is trusted and runs uncapped by default (set [clients.cli] capped = true to cap it too). The caps hold for music an agent starts as well as for the volumes it sets: a play, a resume or a DJ start first brings a room left loud down to its cap. Hand the house to an agent without it playing volume 80 at 2 a.m.
• A DJ that plays what you like. It plays from your own Spotify likes and saves, in whatever genres they span (pop, jazz, hip-hop, soundtracks, classical), puts your own steering ahead of anything it infers, follows mood and time of day, spreads a set across artists, explains every pick, and keeps learning from your likes, dislikes, early skips and full listens. Classical works play whole, every movement in order. Standing preferences of your own — favored and avoided genres, artists and eras, a default energy, explicit tracks, pins and bans — override what it infers, on the CLI, HTTP and MCP.
• One memory-safe binary. Pure Rust 2024 with #![forbid(unsafe_code)], on an owned async stack (no Tokio, no reqwest). Nothing is installed on the speakers and nothing about them is changed.

fsonos discover                                   # every player on the LAN, S1 and S2
fsonos zones                                      # the groups and what each is doing
fsonos play "Living Room" --favorite "Morning"    # a Sonos favorite (or a URI, or --search "bwv 988")
fsonos play Kitchen spotify:album:<id> --next     # after the track playing; --queue for the end
fsonos volume "Living Room" +5                    # set, or change by ±N; --group for the whole group
fsonos eq Kitchen --bass 2 --loudness off         # tone: bass and treble -10..10, loudness on/off
fsonos group Office Kitchen                       # Office joins Kitchen's group
fsonos move "Living Room" Bedroom                 # the music follows you
fsonos dj start "Living Room" --mood calm         # the DJ, from your own Spotify taste
fsonos dj why "Living Room"                       # why it chose this pick
fsonos scene save dinner && fsonos scene apply dinner
fsonos sleep Bedroom 45m                          # fade out, then pause
fsonos schedule add "weekdays 07:30" dj start Kitchen --mood bright
fsonos say "Dinner is ready" --rooms Kitchen,Office
fsonos announce --file clip.wav --rooms Kitchen   # an existing 16-bit PCM WAV
fsonos stream Kitchen --input alsa_output.example.monitor  # Linux desktop audio (Ctrl-C stops)
fsonos undo                                       # put the newest action back
fsonos serve                                      # the daemon: HTTP API on loopback + tailnet, MCP on loopback
fsonos tailscale setup                            # HTTPS for the tailnet via Tailscale Serve (never Funnel)
fsonos doctor                                     # what is wrong, and how to fix it

No speakers handy? Run fsonos sim in one terminal; it prints the --seeds and --routes flags that point every other command (and fsonos mcp) at the virtual house.

On the computer connected to the speaker LAN, install FFmpeg with the libmp3lame encoder and a PipeWire/PulseAudio-compatible audio server. Use pactl list short sources to find your output device's monitor source, then:

fsonos stream Kitchen --input <output-source>.monitor
# The default is --format pulse --input @DEFAULT_MONITOR@.
# Use a dedicated PipeWire/PulseAudio virtual sink to stream only selected apps.
# Ctrl-C ends the capture and restores the previous music if the room still plays this stream.

For macOS with a configured loopback/virtual audio device, choose --format avfoundation --input :<device-index> (check ffmpeg -f avfoundation -list_devices true -i ""). The encoder starts locally; the Sonos player fetches MP3 from a randomly named, read-only endpoint bound only on the LAN interface used to reach that player. There is no public audio service or cloud relay. --daemon-url is not supported for live capture: the computer running stream must reach the players and be reachable by them.

Video lip-sync is not implemented: Sonos normally buffers HTTP-radio playback by seconds; the precise delay also drifts. This command is for live listening rather than in-sync video audio. Automatic calibration and player delay correction remain tracked in issue #2.

Linux discovers FrankenTTS (ftts) first, then espeak-ng. Install the engine and its local models before announcing; FrankenTTS's ftts pull prepares its models. An explicit Piper model takes priority in automatic Linux selection: set piper_model or FSONOS_PIPER_MODEL to a local .onnx file with its .onnx.json sidecar, or pass that model as --voice. macOS keeps using /usr/bin/say by default. fsonos say "Dinner is ready" --voice <voice> selects the backend's voice, and fsonos doctor --only speech identifies the executable.

Configure the data directory's speech.toml on the machine generating speech: the daemon host with --daemon, or the CLI host with --direct. For native FrankenTTS:

backend = "frankentts"
timeout_secs = 180

Alternatively, this custom adapter runs espeak-ng with a default voice:

backend = "command"
command = ["espeak-ng", "--stdin", "-w", "{output}", "-v", "{voice}"]
voice = "en"

The adapter sends text on stdin and substitutes exactly one {output} and optional {voice} inside individual arguments, without shell interpolation. FSONOS_TTS_BACKEND, FSONOS_TTS_VOICE, FSONOS_PIPER_MODEL, and FSONOS_TTS_TIMEOUT_SECS override their file settings. FSONOS_TTS_COMMAND is a JSON argument array (the same shape as command above) and selects the command adapter even when another backend is configured. A request's --voice overrides the configured voice. Synthesis defaults to 180 seconds; allowed timeout_secs values are 1–600.

announce --file uploads the client's WAV bytes to the daemon. POST /announce and the daemon's MCP announce tool accept exactly one of text, chime, or wav_base64 (standard padded base64), plus rooms and volume. Uploaded WAVs must be 16-bit PCM, at most 16 MiB and five minutes; all clips use the same validated media listener, volume caps and restoration. Announcement playback over MCP requires fsonos serve; standalone fsonos mcp has no media listener.

The daemon is the only thing that leaves your LAN, and only onto your tailnet:

fsonos serve            # API on loopback + every tailnet address, MCP on loopback; prints the URLs
fsonos tailscale setup  # Serve: https://<mac>.<tailnet>.ts.net/ (API), :8443/mcp (MCP)
fsonos doctor --only tailscale   # Tailscale up, the MagicDNS name, the daemon answering, Serve in front

• fsonos tailscale setup maps HTTPS on 443 (API) and 8443 (MCP) to the daemon's loopback ports, refuses to run while Funnel is on for either port, and never touches Serve config it didn't make (status, teardown, --dry-run).
• Run the daemon with --tailscale-serve (or FSONOS_TAILSCALE_SERVE=1) behind Serve: it sets Serve up at startup and trusts the login Serve passes along, so each HTTP caller is named by their Tailscale login, a [clients."alice@example.com"] table in policy.toml sets what that person's devices may do, and the action log records who did what. Without the flag, HTTP callers arriving through Serve get the daemon's own loopback rights (loopback-http), and fsonos doctor --only tailscale warns about it. MCP through Serve (:8443) is always one client, loopback-http: narrow it with [clients."loopback-http"]. Callers on the daemon's direct tailnet address are named by Tailscale either way.
• Point an agent at it from any tailnet machine:

claude mcp add --transport http fsonos https://<mac>.<tailnet>.ts.net:8443/mcp
curl https://<mac>.<tailnet>.ts.net/zones

The full walkthrough (launchd, the firewall, tailnet policy grants, the Spotify sign-in) is in docs/DEPLOY.md.

fsonos serve exposes the house as MCP tools (also fsonos mcp over stdio for a local agent, which hands the DJ, scenes, sleep timers and announcements to a running serve):

reads      list_zones · list_rooms · list_players · get_zone · get_zone_state · list_favorites
           search_library · get_queue · recent_plays · recent_actions · get_policy · doctor
           dj_status · dj_moods · events · echo
           list_scenes · get_scene · list_sleep_timers · list_schedules · get_eq
control    play · play_favorite · add_to_queue · pause · resume · next · previous · seek · set_volume
           mute · group · ungroup · move_playback · group_all · set_eq · announce · undo_last
           fade_volume · fade_and_pause
dj         dj_start · dj_skip · dj_stop · dj_steer · dj_feedback · dj_preferences · dj_prefer
           dj_sync · dj_sync_status
house      save_scene · apply_scene · delete_scene · set_sleep_timer · add_schedule
           pause_schedule · resume_schedule · remove_schedule
resources  sonos://zones · sonos://zones/{room} · sonos://dj

events is how an agent keeps up with the house: call it once for the newest event id, then again with since and wait_secs to hear what changes (a track, a volume, a regroup, a player going offline, a logged action, the DJ's picks) as it happens.

Every HTTP route that an MCP tool mirrors has that tool's name as its operation id, so an agent gets the same answer, and the same error code, either way. The API documents itself at GET /openapi.json.

# policy.toml in the data directory. A missing file falls back to the
# [defaults] below only (no quiet hours, room caps or client tables until you
# add them); the other sections here are examples.
[defaults]
max_volume = 70        # per room
max_step = 20          # largest single increase

[quiet_hours]          # local time; may wrap midnight
start = "22:00"
end = "07:00"
max_volume = 25

[rooms."Bedroom"]
max_volume = 40

[clients."alice@example.com"]   # a person behind Tailscale Serve
allow = ["list_zones", "get_zone_state", "play", "set_volume"]

• One choke point: every mutating call on every surface is authorized, snapshotted, carried out under the policy, and logged with who asked, the verdict (allow, clamp or deny) and what happened.
• Edit policy.toml while the daemon runs and its next call obeys the new rules, no restart needed. An edit that does not parse is logged and the last good policy stays in force (fsonos policy check says what is wrong).
• fsonos undo restores the volumes, grouping and what was playing in the zones the newest action changed (--mine: your own newest), and says what it could not. The undo_last tool and POST /undo undo the caller's own newest.
• Callers the daemon cannot identify get read-only tools; the API and MCP listeners refuse wildcard and public binds (unless you pass --allow-unsafe-bind); the HTTP API checks Host and Origin and accepts POST bodies only as JSON.

• Pure where it can be. SSDP, SOAP, GENA and DIDL encoding, planning, the policy, scene diffs, schedules and the DJ's picks are pure functions, unit tested without a network; I/O sits behind narrow traits.
• Events, not polling. The live model follows GENA, renews subscriptions, and resurveys when the house changes; group-wide commands are addressed to the coordinator automatically.
• S1 and S2 together. Each player's generation is recorded, and the code branches only where the protocols differ; S1 and S2 rooms never share a group, and the commands say so instead of failing quietly.
• End to end against a simulator. fsonos-sim serves the real protocols on loopback (queues, favorites, GENA, sleep timers, media fetches, a clock tests can move), and the e2e suite drives the real fsonos binary against it, logging every step as JSON lines.
• No site data. Your rooms, addresses and tokens are learned at runtime and stored locally; the repository holds none of them.

            SSDP / UPnP-SOAP / GENA (your LAN, port 1400)
                              │
          ┌───────────────────▼───────────────────┐
          │  fsonos-proto   discovery · SOAP ·     │
          │                 events · DIDL/URIs     │
          └───────────────────┬───────────────────┘
                              │
          ┌───────────────────▼───────────────────┐   ┌────────────────┐
          │  fsonos-core    inventory · live model │   │ fsonos-spotify │
          │   policy · scenes · schedules · store  │◄──┤ library + DJ   │
          └───────────────────┬───────────────────┘   └───────┬────────┘
                              │                               │
   ┌──────────────┬───────────┼───────────────┐       Spotify Web API
   ▼              ▼           ▼               ▼       (read-only, your account)
fsonos-api     fsonos-mcp   fsonos-cli    fsonos-tailscale
(HTTP API)    (MCP tools)   (`fsonos`)    (tailnet presence)

Built on the author's Rust stack: asupersync (async runtime), fsqlite (local store), fastmcp_rust, and fastapi_rust. The full design is in COMPREHENSIVE_PLAN_FOR_FRANKENSONOS.md.

No prebuilt binaries yet (pre-release). Build from source:

git clone https://github.com/Dicklesworthstone/frankensonos
cd frankensonos
cargo build --release      # the toolchain is pinned in rust-toolchain.toml
cargo test --workspace     # unit, golden and end-to-end tests (no speakers needed)

The binary is target/release/fsonos. Run it on a machine on the same LAN as your speakers; for access from anywhere, that machine also joins your tailnet. fsonos setup takes it from there. To run the daemon at boot, see docs/DEPLOY.md (launchd plus Tailscale).

Shipped: discovery and topology across S1 and S2, coordinator-addressed control and grouping, the GENA live model and self-healing, the DJ (drawing on everything you save or like on Spotify, plus your top tracks, recent plays and own playlists, in any genre) with your standing preferences, steering, explanations and feedback, scenes, sleep timers and schedules, announcements, the house policy with quiet hours, the action log and undo, doctor and setup, the simulator, and the CLI, HTTP API and MCP server, on loopback and your tailnet. The CLI goes through a running daemon when there is one, with room-name completions for zsh, bash and fish, and the daemon serves a web remote (rooms, now playing with album art, volume and mute, pause everywhere, the DJ and scenes) to any device on your tailnet. The daemon keeps the DJ's library fresh on its own, and fsonos dj sync refreshes it on demand.

Next:

• Real-hardware CI, then 0.1.0 with prebuilt binaries.

FrankenSonos controls hardware you own, on your own network, and reads your own Spotify library with your own credentials (read-only, PKCE). It never writes to device firmware or changes anything on the speakers, and it handles no secrets beyond your own local OAuth cache. This repository is public and contains no site data (addresses, serials, tokens, room lists); all of that is learned at runtime and kept locally. The full boundary is in docs/SCOPE.md.

• It is pre-release: simulator-verified end to end, with opt-in live tests but no real-hardware CI yet. Expect interfaces to change until 0.1.0.
• Sonos plays Spotify through its own music-service integration; the Spotify Web API is only used to read your library. Spotify must be linked once in each household's Sonos app, and the per-household render parameters are learned from your own Sonos favorites. Rendering on legacy S1 players is still being confirmed on real hardware.
• The DJ's taste comes from your liked tracks and saved albums, plus your top tracks, recent plays and own playlists, leaning toward the artists you follow and play most (read-only scopes you grant at sign-in; without them it uses your library alone, and doctor says so). dj steer, standing preferences and moods.toml are your overrides. Spotify gives new apps no audio features, so a song's energy comes from its artists' genre tags, and a classical movement's from its tempo marking. New saves and likes reach the DJ at the daemon's next daily read of your library, or at once with fsonos dj sync.
• You run the daemon on a machine on your speaker LAN (a Mac mini, a Pi, a NAS). The deploy docs describe macOS launchd; Linux speech uses a locally installed engine and models.
• A sleep timer fades only through the daemon: the CLI's when one is running, HTTP and MCP. fsonos sleep --direct, or fsonos sleep with no daemon, sets the speaker's own timer, which pauses without a fade.
• Spotify artists, podcast episodes and shows don't play yet; tracks, albums and playlists do.
• The API and MCP server have no authentication of their own: Tailscale is the boundary, and it names the HTTP API's callers (Tailscale's WhoIs on a direct tailnet listener, or the login header behind Serve when the daemon runs with FSONOS_TAILSCALE_SERVE) for the house policy's [clients] tables. A caller Tailscale can't name, and any MCP caller off loopback, is read-only.

Does this replace the Sonos app? For day-to-day control, grouping, the DJ, scenes, schedules and announcements, yes. You still link Spotify once in the official app so Sonos can render it.

Does it modify my speakers or their firmware? No. It sends them the same local control requests a controller always has. Nothing is flashed or changed on the devices.

Why can't it just use the Spotify API to play music? The Spotify Web API cannot target a Sonos renderer. Sonos plays Spotify through its own integration, so FrankenSonos builds the right x-sonos-spotify: request for your household, from parameters learned from your own favorites.

S1 and S2 at once? Yes. Both households appear together, every command addresses the right one, and Room@S1 / Room@S2 picks between rooms with the same name.

Can my AI agent control it from another city? Yes, over your Tailscale tailnet: the agent reaches the daemon, and the daemon reaches the speakers on the LAN. The house policy bounds what it may do, and undo puts it back.

Can I try it without Sonos speakers? Yes: fsonos sim runs a virtual S1 and S2 house on loopback, and every command and the MCP server work against it.

What streaming services work? Spotify first, plus anything you have saved as a Sonos favorite (stations, playlists, other services). The control layer is source-agnostic.

Please don't take this the wrong way, but I do not accept outside contributions for any of my projects. I simply don't have the mental bandwidth to review anything, and it's my name on the thing, so I'm responsible for any problems it causes; thus, the risk-reward is highly asymmetric from my perspective. I'd also have to worry about other "stakeholders," which seems unwise for tools I mostly make for myself for free. Feel free to submit issues, and even PRs if you want to illustrate a proposed fix, but know I won't merge them directly. Instead, I'll have Claude or Codex review submissions via gh and independently decide whether and how to address them. Bug reports in particular are welcome. Sorry if this offends, but I want to avoid wasted time and hurt feelings. I understand this isn't in sync with the prevailing open-source ethos that seeks community contributions, but it's the only way I can move at this velocity and keep my sanity.

MIT License with an OpenAI/Anthropic rider; see LICENSE. (LicenseRef-MIT-OpenAI-Anthropic-Rider; not plain MIT.)