| ๐ Observability | ๐ธ๏ธ Agent Tracing | ๐ LLM Routing |
|---|---|---|
| ๐ฐ Cost & Latency Tracking | ๐ Datasets & Fine-tuning | ๐๏ธ Automatic Fallbacks |
Docs โข Changelog โข Bug reports โข See Helicone in Action! (Free)
Helicone is an AI Gateway & LLM Observability Platform for AI Engineers
- ๐ AI Gateway: Access 100+ AI models with 1 API key through the OpenAI API with intelligent routing and automatic fallbacks. Get started in 2 minutes.
- ๐ Quick integration: One-line of code to log all your requests from OpenAI, Anthropic, LangChain, Gemini, Vercel AI SDK, and more.
- ๐ Observe: Inspect and debug traces & sessions for agents, chatbots, document processing pipelines, and more
- ๐ Analyze: Track metrics like cost, latency, quality, and more. Export to PostHog in one-line for custom dashboards
- ๐ฎ Playground: Rapidly test and iterate on prompts, sessions and traces in our UI.
- ๐ง Prompt Management: Version prompts using production data. Deploy prompts through the AI Gateway without code changes. Your prompts remain under your control, always accessible.
- ๐๏ธ Fine-tune: Fine-tune with one of our fine-tuning partners: OpenPipe or Autonomi (more coming soon)
- ๐ก๏ธ Enterprise Ready: SOC 2 and GDPR compliant
๐ Generous monthly free tier (10k requests/month) - No credit card required!
Quick Start โก๏ธ
-
Get your API key by signing up here and add credits at helicone.ai/credits
-
Update the
baseURLin your code and add your API key.import OpenAI from "openai"; const client = new OpenAI({ baseURL: "https://ai-gateway.helicone.ai", apiKey: process.env.HELICONE_API_KEY, }); const response = await client.chat.completions.create({ model: "gpt-4o-mini", // claude-sonnet-4, gemini-2.0-flash or any model from https://www.helicone.ai/models messages: [{ role: "user", content: "Hello!" }] });
-
๐ You're all set! View your logs at Helicone and access 100+ models through one API.
Self-Hosting Open Source LLM Observability
Docker
Helicone is simple to self-host and update. To get started locally, just use our docker-compose file.
# Clone the repository git clone https://github.com/Helicone/helicone.git cd docker cp .env.example .env # Start the services ./helicone-compose.sh helicone up
Helm
For Enterprise workloads, we also have a production-ready Helm chart available. To access, contact us at enterprise@helicone.ai.
Manual (Not Recommended)
Manual deployment is not recommended. Please use Docker or Helm. If you must, follow the instructions here.
Architecture
Helicone is comprised of five services:
- Web: Frontend Platform (NextJS)
- Worker: Proxy Logging (Cloudflare Workers)
- Jawn: Dedicated Server for serving collecting logs (Express + Tsoa)
- Supabase: Application Database and Auth
- ClickHouse: Analytics Database
- Minio: Object Storage for logs.
Integrations ๐
Inference Providers
| Integration | Supports | Description |
|---|---|---|
| AI Gateway | JS/TS, Python, cURL | Unified API for 100+ providers with intelligent routing, automatic fallbacks, and unified observability |
| Async Logging (OpenLLMetry) | JS/TS, Python | Asynchronous logging for multiple LLM platforms |
| OpenAI | JS/TS, Python | Inference provider |
| Azure OpenAI | JS/TS, Python | Inference provider |
| Anthropic | JS/TS, Python | Inference provider |
| Ollama | JS/TS | Run and use large language models locally |
| AWS Bedrock | JS/TS | Inference provider |
| Gemini API | JS/TS | Inference provider |
| Gemini Vertex AI | JS/TS | Gemini models on Google Cloud's Vertex AI |
| Vercel AI | JS/TS | AI SDK for building AI-powered applications |
| Anyscale | JS/TS, Python | Inference provider |
| TogetherAI | JS/TS, Python | Inference provider |
| Hyperbolic | JS/TS, Python | Inference provider |
| Groq | JS/TS, Python | High-performance models |
| DeepInfra | JS/TS, Python | Serverless AI inference for various models |
| Fireworks AI | JS/TS, Python | Fast inference API for open-source LLMs |
Frameworks
| Framework | Supports | Description |
|---|---|---|
| LangChain | JS/TS, Python | Use AI Gateway with LangChain for unified provider access |
| LlamaIndex | Python | Framework for building LLM-powered data applications |
| LangGraph | Python | Build stateful, multi-actor applications with LLMs |
| Vercel AI SDK | JS/TS | AI SDK for building AI-powered applications |
| Semantic Kernel | C#, Python | Microsoft's AI orchestration framework |
| CrewAI | Python | Framework for orchestrating role-playing AI agents |
| ModelFusion | JS/TS | Abstraction layer for integrating AI models into JavaScript and TypeScript applications |
| PostHog | JS/TS, Python, cURL | Product analytics platform. Build custom dashboards. |
| RAGAS | Python | Evaluation framework for retrieval-augmented generation |
| Open WebUI | JS/TS | Web interface for interacting with local LLMs |
| MetaGPT | YAML | Multi-agent framework |
| Open Devin | Docker | AI software engineer |
| Mem0 EmbedChain | Python | Framework for building RAG applications |
| Dify | No code required | LLMOps platform for AI-native application development |
This list may be out of date. Don't see your provider or framework? Check out the latest integrations in our docs. If not found there, request a new integration by contacting help@helicone.ai.
Contributing
We โค๏ธ our contributors! We warmly welcome contributions for documentation, integrations, costs, and feature requests.
If you have an idea for how Helicone can be better, create a GitHub issue.
License
Helicone is licensed under the Apache v2.0 License.
Additional Resources
-
LLM Cost API: We have the largest open-source API pricing database with 300+ models and providers such as OpenAI, Anthropic and more. Start querying here.
-
Data Management: Manage and export your Helicone data with our API or access it with our MCP server.
- Guides: ETL, Request Exporting
-
Data Ownership: Learn about Data Ownership and Autonomy
For more information, visit our documentation.