CopilotKit/OpenIntelligentUI

The 54-second launch film: ask, explore a 3D explanation, compare options, use a calculator, and follow a coastal itinerary. These are rendered launch scenes; the app runs the agent for each request.

Ask a question, explore an idea, compare your options, or make a tool for the moment. The agent chooses a direct text answer, a native component, or a custom interactive UI based on what helps you accomplish the task.

• Understand — explore a mechanism, step through an explanation, or change a variable.
• Compare — examine criteria, assumptions, and tradeoffs side by side.
• Make a tool — use a calculator, planner, or interactive model with working controls.

The interface opens directly to chat. Type your request or choose a starter suggestion to begin an agent run. Text, native components, and generated interactive answers appear in the conversation. Sample numbers are illustrations, not live business or weather data.

From the launch film: explanations, comparisons, working tools, and maps in one conversation.

Rotate an airplane with labeled pitch, roll, and yaw axes. Change the angles, play a smooth demonstration, or reset the view.

3D plane · 4 seconds.

Compare exact values in a table, then ask a follow-up to visualize the differences. Jev selects A2UI for basic tables and Open Generative UI for charts and more complex visuals.

Tables and charts · 8 seconds. Plan names and prices are illustrative.

Split a bill with controls for the total, tip, and group size. Calculations update as you change the inputs.

Bill splitter · 5 seconds.

Explore a coastal itinerary with numbered pins and destination photos. The app uses live USGS tiles, credits its photos, and lets you pause, replay, or select a stop. Connections illustrate the itinerary; they are not verified driving directions.

Coastal map · 4 seconds.

Scene gallery

3D explanations	Tables → charts
Explore aircraft rotation with labeled axes and sliders.	Compare exact values, then visualize them with a follow-up.

Interactive tools	Maps
Change the group size and tip; the calculation updates.	Follow a coastal itinerary with map pins and destination photos.

• Animate a coastal trip — a real USGS map with numbered pins, credited destination photos, and a six-second pin-drop sequence. Pause, replay, or select a stop. Connections are illustrative, not verified driving directions.
• Pitch, Roll & Yaw — rotate a 3D airplane with labeled axes, angle controls, smooth demonstrations, and reset.
• Find your kind of weekend — compare trip ideas and adjust your preferences.
• Split the bill fairly — change the total, tip, and group size in a working calculator.

These starters run the agent; they are not prerecorded responses. Jev chooses the presentation, so outputs can vary. Map tiles and destination photos require network access. Streaming answers preserve your reading position, reveal text as it enters view, and show the copy action on hover or keyboard focus (always available on touch).

Prerequisites: Node.js 22+, pnpm 9+, Python 3.12+, and uv.

git clone https://github.com/CopilotKit/OpenIntelligentUI.git
cd OpenIntelligentUI
make setup
# Optional: set shared OPENAI_API_KEY and TYPESAFE_API_KEY in apps/agent/.env
# Or use API keys in the chat header after starting the app
make dev

Open the app. The agent health endpoint confirms the agent service is running. See Getting started for configuration and verification.

The default model is chat-latest, OpenAI’s documented alias for the latest ChatGPT Instant model, requiring OPENAI_API_KEY. Jev (jev-latest, requiring TYPESAFE_API_KEY) selects the renderer and visualization for each user turn. Basic tables use A2UI; charts, diagrams, calculators, and maps use Open Generative UI. Provider failures are surfaced instead of silently substituting another router or model. See Visualization routing.

The chat asks for both keys before sending your first message or example prompt and keeps your draft while you set them up. Open API keys in the chat header, enter an OpenAI key and a Jev key, then choose Test connection and Save keys. The connection test makes a small request to each provider. Your keys use chat-latest for answers and jev-latest for visualization routing, with usage billed to your provider accounts.

Keys stay in browser memory until refresh or Clear keys; the app does not save them in browser storage. Requests send them through the application server to their respective providers. They are kept out of chat state and checkpoints, and hosted LangSmith tracing is disabled for requests using your keys. Only use this feature on a deployment whose operator you trust. Saving or clearing keys starts a new chat; New chat keeps your keys for the current session.

The agent can run without shared provider keys. Visitors must then supply both keys before chatting. Configured server credentials remain available to direct API clients that omit visitor keys; the chat interface requires visitors to add their own keys.

The Python Deep Agent uses a task-first system prompt and focused skills. CopilotKit carries the agent stream to the Next.js app. Native components handle supported structured tasks; generateSandboxedUi streams custom HTML, CSS, and JavaScript into an isolated iframe.

Custom UI parameters must arrive in this order:

initialHeight → placeholderMessages → css → html → jsFunctions → jsExpressions

Local controls run inside the sandbox. A user-clicked follow-up can send selected values through the validated host bridge and start another agent turn. Prior outputs remain separate conversation artifacts; there is no cross-call patch API. See Product and agent behavior and Architecture.

apps/app/                Next.js + CopilotKit frontend
apps/agent/              Python Deep Agent + FastAPI
apps/mcp/                Optional standalone MCP server
packages/design-system/  Shared theme, SVG, and form styles

make dev-app     # Frontend
make dev-agent   # Agent
make dev-mcp     # Optional MCP server
pnpm test       # JavaScript workspace tests
uv run --directory apps/agent pytest
make lint
make build

The launch workflow includes browser checks and a real provider smoke test. Tests and builds alone do not verify model access or a deployment.

The optional MCP server exposes skill resources, prompt templates, and assemble_document. The assembler returns an HTML document as text; a compatible host must render it and implement its bridge. It is distinct from the web app's streaming Websandbox integration.

See the MCP server guide for HTTP, stdio, and Docker configuration.

Built with CopilotKit. The openGenerativeUI runtime API and generateSandboxedUi tool power the streaming experience.

• Documentation index
• Bring these patterns to your app
• Source repository
• Telemetry and how to turn it off

See Contributing for the development workflow and the documentation index for guides to the agent, rendering, and deployment.

MIT.