The Sesame Robot Project
Greetings, from your new best friend.
Sesame is an accessible Open-Source robotics project based on the ESP32 microcontroller system, with an emphasis on expression and movement. This project is designed for makers and engineers of all skill levels! Sesame offers a dynamic platform designed to start working with walking robots. To build a sesame robot, you will need basic soldering skills, $50-60 in hardware components, access to a 3D printer, and a basic understanding of Arduino IDE.
This repository contains the CAD design files, STL files, build and wiring guides, and the base/expanded firmware for the ESP32-based controller. There is also some included debugging firmware that may be helpful in getting your Sesame up and running.
Features
- Quadruped Design: Uses 8 servo motors (2 per leg) to achieve roughly 8 total degrees of freedom.
- Emotive Display: Features a 128x64 OLED screen acting as a reactive face that syncs with movement.
- Fully Printable: Designed entirely for 3D printing in PLA with minimal supports.
- Network Connectivity: Connect to your WiFi network for remote control and API access.
- JSON API: RESTful API for programmatic control from Python, JavaScript, and more.
- Conversational Faces: Expressive emotion library with talk variants for voice assistant projects.
- Sesame Studio: New animation composer software to easily create custom movements.
- Sesame Companion App: Python application for voice control and advanced interactions.
- Serial CLI: Control the robot and trigger animations via a Serial Command Line Interface or the web UI.
- Pre-programmed Emotes: Includes animations for Walking, Waving, Dancing, Pointing, Resting, and more.
Watch the launch video on YouTube
Getting Started
Follow these steps to build your own Sesame Robot:
1. Gather Parts
Check the Bill of Materials (BOM) for a complete list of required electronics and hardware.
- Microcontroller: Lolin S2 Mini (recommended for DIY builds), Sesame Distro Board V3 (Current, pre-flashed, supports Bambu Lab battery), V2 (legacy, USB-only), or ESP32-DevKitC-32E with Distro Board V1 (legacy)
- Actuators: 8x 180 Degree MG90 Servos
- Power: 5V 3A source (USB-C PD for S2 Mini and V2 Distro Board, or battery + buck converter; see BOM for the Bambu Lab 14500 7.4V 800mAh Li-ion Battery option)
2. Print Parts
Download the STLs and follow the Printing Guide.
- Designed for PLA
- Minimal supports required
3. Build & Wire
Follow the Build Guide and Wiring Guide to assemble the frame and connect the electronics.
4. Flash Firmware
Upload the code from the Firmware Directory.
- Requires Arduino IDE
- Configure WiFi AP settings
5. Create Animations
Use Sesame Studio to visually design poses and sequences for your robot.
Software & Firmware
Sesame Studio
Sesame Studio is a standalone desktop application included in software/sesame-studio/. It allows you to:
- Visually pose the robot using a schematic interface.
- Generate C++ code for servo angles automatically.
- Sequence frames into complex animations.
Sesame Simulator
The Sesame Simulator, created by Jay Li, is a Rust-based 3D simulation environment for testing Sesame's movements and kinematics in a virtual space. It features:
- Physics-based Simulation: Test walking and balance without hardware.
- Web-based Interface: Run the simulator directly in your browser.
- URDF Integration: Accurate modeling of Sesame's physical properties.
Sesame ML Tools
ML tools Created by Luke Hollis for Sesame, world model for online planning, policy eval, and RL finetuning.
- Robot description: Self-contained printable STLs, CAD-derived MJCF, validated URDF, camera/IMU/foot frames, primitive contact geometry, masses and actuator limits
- Tasks: Stand, challenged recovery, commanded locomotion, and goal/obstacle navigation
- Policy evaluation: Multi-seed JSON/CSV reports, strict task success, domain parameters, tracking/fall/collision/energy/latency metrics, MP4/GIF and onboard/split views
Sesame Companion App
The Sesame Companion App is a Python-based application that enables advanced control and interaction with your robot over your local network. It leverages the new JSON API and network mode features to provide:
- Voice Assistant Integration: Control Sesame with voice commands and see real-time emotional expressions.
- Remote Control: Command your robot from anywhere on your local network.
- Face Control: Change expressions dynamically based on conversation or context.
- API Examples: Reference implementation for building your own integrations.
The Companion App works with robots running the latest firmware with network mode enabled.
> Go to Sesame Companion App Repository
Firmware
The ESP32 firmware (sesame-firmware-main.ino) handles the kinematics, face display, and WiFi control interface.
- Web UI: Control the robot from your phone via the built-in Access Point.
- Custom Faces: Add your own bitmaps (guide in firmware docs).
Contributing
This robot is a platform for building new features, cosmetics, tools, and ideas. Since the current firmware is a basic implementation, pull requests are very welcome for:
- Kinematics improvements
- New animations
- Improved Web UI/UX
- Sensor integration (Ultrasonic, Gyro, etc.)
I would also love to see forks of this project with new hardware, software, faces, etc. Be sure to send me a message if you end up building one, and I might feature you on my website or channel!
Created by Dorian Todd. Need help with your Sesame Robot? Send me a message on Discord, my username is "starphee"