NomaDamas/CozyClay

Block a scene, pose the cast, cut the camera — in a browser tab.

Created and maintained by Doyun at NomaDamas.

Try it in the browser · Quick start · Features · AI control · Workflow · Controls · Docs · Issues

CozyClay is a browser-based previs studio built with Three.js and React Three Fiber. Block a scene, pose the cast, cut the camera on a timeline, then hand the same shot to an AI video model (Seedance, Kling, Veo, or your own) as a first frame, a reference clip, or a prompt — all from one local workspace. The keyframe pack's greybox clip is the input Seedance 2.5 documents as white-model control: "Use the white-model reference video as the sole guide for camera movement, pacing, framing, subject motion, and blocking" (seed.bytedance.com/en/seedance2_5). MiniMax H3, Wan 3.0, LTX Desktop and fal render-to-real take the same plain RGB clip.

npx cozyclay

That is the whole install. Not sure yet? Try it in the browser first — a seven-step camera tutorial on a live scene — then keep going on your machine with the same set:

npx cozyclay --scene city-block

The studio ships seeded with a pre-generated motion clip, so you can scrub the timeline, drive the cameras and draw a dolly rail straight away — generating new motion is optional and uses the Kimodo bridge when configured.

A project is either a Storyboard or an Animation. Pick it on the start screen (the badge in the top bar shows which one you are in); a project keeps its mode when saved and reopened.

Start in Storyboard when you are deciding what the shots are, then open the idea as an Animation when it needs motion.

You need Node.js 22.19 or newer, npm or bun, and a Chromium-based browser.

npx cozyclay
# or
bunx cozyclay

That downloads the built studio and opens it at http://127.0.0.1:5180/app/. Nothing to compile, no dependency tree to install. Useful flags: --port 5200, --no-open, --no-motion, --scene city-block (start on the bundled starter scene instead of an empty room; the first-run dialog offers the same under Start from a scene, and a .cclayproject downloaded from the browser tutorial opens with Open a project).

A global install gives you cclay, the same command with less typing. Once a day the launcher checks npm for a newer release and prints a one-line notice after the studio is up; it stays quiet when you're current or offline. cclay update installs the latest release, and --no-update-check skips the check entirely.

New to the camera? The seven-step tutorial runs inside the Studio on the City Block set: Settings ▾ → Camera tutorial, or open http://127.0.0.1:5180/app/?tutorial=camera. Each step points at the control it needs and completes only when you actually make the move.

git clone https://github.com/NomaDamas/CozyClay.git
cd CozyClay
npm install
npm run dev

Open http://127.0.0.1:5180/app/ for the Studio (the root redirects there) and http://127.0.0.1:5180/workflow/ for the Workflow canvas. npm run dev starts the studio together with its local Kimodo bridge once CCLAY_KIMODO_HOST points at a GPU box; without that variable it starts the studio alone and says so. npm run dev:ui starts the browser UI alone in every case.

Generating new motion needs Kimodo on a machine you can reach. Point the Studio at it and install the worker once:

CCLAY_KIMODO_HOST=user@your-gpu-box npm run kimodo:setup
CCLAY_KIMODO_HOST=user@your-gpu-box npx cozyclay

The installer detects OS, architecture, RAM and CUDA and picks the best-supported backend for that host — kimodo-mlx on a big Apple Silicon machine, kimodo.cpp with Metal or CPU otherwise, the upstream PyTorch stack on NVIDIA. An SSH-accessible NVIDIA box is the classic target and the only route with the full feature set; the same box runs GVHMR for video and photo mocap. Routes, overrides and local-only limits are in docs/kimodo-setup.md.

Three surfaces drive the same scene, live, through one hub: a chat panel in the Studio, an MCP server for external assistants, and a terminal CLI.

View ▾ → Panels → Agent panel (or Cmd/Ctrl+B) opens a chat column that signs in with your ChatGPT account (Codex OAuth; the token lives in ~/.config/cozyclay/) and works the scene in front of you:

"Put a detective and a courier in an alley, give me a low wide profile shot, then make her stand up from the chair, sprint, and trip."

• Every change comes back as a receipt that highlights the Inspector and hierarchy rows it touched, and Cmd/Ctrl+Z takes it back like any other edit.
• The conversation is stored under ~/.config/cozyclay/agent-sessions/ and resumes across reloads and restarts; History lists earlier sessions.
• Paste or drop up to four images into the composer — a reference frame, a storyboard panel — and they go with the turn.
• The agent has nine Studio tool families, from inspect_studio and arrange_characters to patch_elements, which sets any declared authored field (a character's tint, a shot's target model, the key light) by path. How it turns a pasted video prompt into blocking — what it asks, infers, or leaves empty — is the rule in docs/agent-prompt-to-scene.md.

The panel is not tied to ChatGPT. Models come from six providers: ChatGPT (your Codex sign-in), Anthropic, OpenAI, Google Gemini, OpenRouter and CLIProxyAPI. A model id reads provider/model, so you pick one in the dropdown and that is where the turn goes. The five API-key providers take a key from an environment variable (ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY or GOOGLE_API_KEY, OPENROUTER_API_KEY, CLIPROXY_API_KEY) or from ~/.config/cozyclay/providers.json, written at mode 0600 and never echoed back by any route; ChatGPT still signs in the way it always did. On the Workflow canvas you can steer a turn while it runs instead of stopping it. Conversations are saved in a new transcript format, so sessions recorded before this version are ignored rather than converted: the old files stay on disk, unread. The provider table, the key routes, the steer route and the session format are in docs/agent-cli.md.

The agent runs on pi, an open-source agent harness: @earendil-works/pi-ai and @earendil-works/pi-agent-core (MIT) are the only runtime dependencies the package declares, and the sidecar loads them lazily, when a turn starts.

The same panel sits on the right of the Workflow canvas, where it builds and runs canvas nodes.

The studio ships an MCP server, so any MCP client can drive it — the same scene, the same viewport:

{
  "mcpServers": {
    "cozyclay": {
      "command": "npx",
      "args": ["-y", "cozyclay", "mcp"]
    }
  }
}

Drop that into claude_desktop_config.json (or any MCP client config) and restart the client. The first run automatically installs the MCP SDK's 95-package tree; opening the studio never waits on it, so those dependencies are fetched only when you actually want the server.

• Editor open? Tool calls move the visible viewport — camera, cast, set, generated motion, prompt blocks on the timeline.
• No editor? Scene and project tools run headless: block scenes, derive film vocabulary ("wide shot · right profile · knee level · 24mm"), render AI video prompts, and write .cclayproject files. capture_frame, set_prompt_blocks, generate_motion, and apply_batch require the live editor.

Tools, transports and the live-control protocol are documented in mcp/README.md.

cclay live is a small CLI that reads the scene, places and moves things, frames and keys shots, patches declared fields (cclay live patch --target stage --set '{"keyLight.intensity":2.4}'), verifies the result with a capture PNG, and undoes by receipt — one JSON object per command, stable error codes, no browser of your own required (only the studio tab itself). It is the intended surface for a coding agent that has to manage a running Studio. The full guide, with three worked sessions against a real editor, is in docs/agent-cli.md.

The Workflow canvas at http://127.0.0.1:5180/workflow/ is a node editor around the Studio: a CozyClay Scene node with a live viewport that previews the shot camera, a Shot Prompt node that builds a structured prompt from the capture, Image nodes (versions, A/B, pinned references) and a Video node that runs through your own ComfyUI (COZYCLAY_COMFY_URL) or fal (FAL_KEY) — bring your own key, nothing is proxied. Graphs are saved in the browser and execute locally.

Every control's home is recorded in docs/studio-ui-ia.md, with the rule behind each placement.

Longer reads on the site: Greybox to AI video, Seedance 2.5 camera control, Camera control for AI video, Previs software compared.

Ad-hoc browser QA, while a dev server is available (the browser opens the studio at /app/):

npm run qa:browser -- node <qa-script>

Found something broken, or want a feature? Open an issue — bug reports with a repro are the most useful thing you can send. Contributions are accepted under AGPL-3.0-or-later.

Repository hygiene. Generated motion archives, QA output, build output, logs and local runtime artifacts are not source files and must not be committed. Keep tools/ardy/out/, artifacts/, dist/, .gjc/ and .npz files local.

Every runtime library except the two agent packages (@earendil-works/pi-ai, @earendil-works/pi-agent-core) intentionally lives in devDependencies, because the published npm package ships the prebuilt dist/, so npx cozyclay must not install the studio's dependency tree.

Anonymous usage counts, no cookies, no recordings, never your project content — on the hosted site via PostHog and in the official npm package, which prints the disclosure once on first launch. Source checkouts, forks, dev servers, CI and tests send nothing.

cclay telemetry status
cclay telemetry off

COZYCLAY_TELEMETRY=0 or DO_NOT_TRACK=1 disables collection for a launch; Settings ▾ → Privacy is the same switch inside the Studio. The event list, what each identifier is, how internal QA traffic is marked and the first-edit definition are in docs/privacy.md.

Installing a GPU motion backend is the hard part, so cozyclay.org also runs a queued demo: a visitor writes one prompt, gets a ticket link, and a GPU box owned by the maintainer generates the motion and uploads it — the result opens in the studio itself. The composer pages, the Cloudflare Worker queue API and the GPU-box poller all live in this repository; the queue policy and runbooks are in workers/api/README.md.

GNU Affero General Public License v3.0 or later — see LICENSE and the transition details in LICENSING.md. Modified network services must offer their users the corresponding source. Third-party projects retain their own licenses and copyright; see THIRD_PARTY_NOTICES.md.

The hosted demo worker may use an externally installed NVIDIA ARDY runtime. ARDY is a separate third-party project owned and maintained by NVIDIA; it is not included in this repository, and CozyClay is not affiliated with or endorsed by NVIDIA. The local Studio uses Kimodo instead.