─── B ───
The Soul-Wired Desktop Companion
B is not just an AI : he is a digital lifeform designed to live on your desktop. Built with a soul-first architecture, B observes your workflow, listens with neural precision, and interacts through a high-performance, glassmorphic overlay.
Explore the Vision • Architecture • Getting Started • Contributing • Privacy
Demo
B in action - watch him observe, think, and respond.
The Vision
Traditional assistants wait for a command. B waits for a moment.
B is designed as a proactive desktop companion. Using a 60fps event-driven central nervous system, B synchronizes his emotional state with your environment. He sees your screen semantically, tracks your active focus, and intervenes only when he has something truly valuable to contribute.
- Silent by Default : B respects your deep work state.
- Emotionally Aware : A complex internal state machine drives expressions and curiosity.
- Agentic Autonomy : B doesn't just respond; he thinks, wonders, and observes.
Architecture Overview
B is built on a centralized asynchronous pub/sub event bus : the EventBus. Every module communicates exclusively through this bus. No module knows about any other module. This strict decoupling makes the system testable, maintainable, and resilient.
flowchart TB
subgraph P["Perception Layer"]
WT["WindowTracker"]
SS["SemanticSensor"]
VS["VisionSensor OCR"]
end
subgraph C["Cognitive Layer"]
CE["CognitiveEngine"]
SM["StateMachine"]
AE["AutonomyEngine"]
end
subgraph O["Output Layer"]
VE["VoiceEngine"]
FR["FaceRenderer"]
CB["ChatBubble"]
KE["KinematicsEngine"]
end
subgraph I["Core Infrastructure"]
EB["EventBus"]
end
WT -- "active_window_changed" --> EB
SS -- "context_updated" --> EB
VS -- "context_updated" --> EB
EB -- "context_updated" --> CE
EB -- "tick" --> SM
EB -- "tick" --> KE
CE -- "b_spoke" --> EB
EB -- "b_spoke" --> VE
EB -- "b_spoke" --> FR
EB -- "b_spoke" --> CB
CE -- "b_move_request" --> EB
EB -- "b_move_request" --> KE
AE -- "trigger_proactive_thought" --> EB
EB -- "trigger_proactive_thought" --> CE
System Workflow
The data flows through B in a deterministic pipeline: Perception → Cognition → Expression.
sequenceDiagram
participant WT as WindowTracker
participant SS as SemanticSensor
participant VS as VisionSensor
participant CE as CognitiveEngine
participant VE as VoiceEngine
participant FR as FaceRenderer
WT->>SS: active_window_changed
activate SS
SS->>SS: UIA Tree Walk
alt Extraction Success
SS->>CE: context_updated
else Failure / Cooldown
SS->>VS: semantic_extraction_failed
VS->>VS: OCR Capture
VS->>CE: context_updated
end
deactivate SS
CE->>CE: LLM Inference
CE->>VE: b_spoke (Text + Emotion)
activate VE
VE->>VE: TTS + DSP Vocoder
VE->>FR: speaking_start
VE->>VE: Audio Output
VE->>FR: speaking_end
deactivate VE
CE->>FR: b_move_request (Spatial)
Internal Module Structure
Core Infrastructure
| Module | File | Responsibility |
|---|---|---|
| EventBus | core/bus.py |
Thread-safe pub/sub message broker. All inter-module communication flows through this. |
| main.py | main.py |
Boot sequence : instantiates all modules, starts the 60fps tick timer, registers global hotkeys. |
Perception Layer (Sensors)
| Module | File | Responsibility |
|---|---|---|
| WindowTracker | sensors/window_tracker.py |
Hook-based active window change detection. Fires when the user switches focus. |
| SemanticSensor | vision/semantic.py |
UIA-based DOM/window tree walking. Extracts structured content with quality scoring and adaptive cooldown. |
| VisionSensor | vision/mss_capture.py |
OCR fallback pipeline using MSS + Tesseract for frameworks incompatible with UIA. |
Cognitive Layer (The Brain)
| Module | File | Responsibility |
|---|---|---|
| CognitiveEngine | brain/llm.py |
LLM inference orchestration (Groq cloud or local llama-cpp). Manages context, history, spatial mapping, and streaming token parsing. |
| StateMachine | brain/soul.py |
B's emotional state : blinking, resting, conversing. Real-time stream buffer that parses LLM output into sentences. |
| AutonomyEngine | brain/autonomy_loop.py |
Proactive thought scheduling : decides when B should speak unprompted based on context quality and timing. |
Output Layer (Expression)
| Module | File | Responsibility |
|---|---|---|
| VoiceEngine | audio/speaker.py |
Piper ONNX TTS with DSP vocoder chain (pitch shift, bitcrush, chorus) for robotic modulation. |
| FaceRenderer | ui/face.py |
PyQt6 QPainter-based hardware-accelerated face rendering at 60fps. |
| ChatBubble | ui/chat.py |
Glassmorphic chat overlay that displays B's spoken text. |
| KinematicsEngine | physics/kinematics.py |
Physics-based movement with Bezier path interpolation and easing curves. |
| EarsSensor | audio/ears.py |
Speech-to-text via Faster-Whisper with neural VAD. |
Data Flow
flowchart LR
subgraph Input["Input"]
A["User Types"]
B["User Speaks"]
C["Screen Changes"]
end
subgraph Process["Processing"]
D["EventBus"]
E["CognitiveEngine"]
F["StateMachine"]
end
subgraph Output["Output"]
G["FaceRenderer"]
H["VoiceEngine"]
I["ChatBubble"]
J["Kinematics"]
end
A -- types --> D
B -- speaks --> D
C -- changes --> D
D -- routes --> E
E -- drives --> F
F -- animates --> G
F -- speaks --> H
F -- displays --> I
E -- moves --> J
Request Lifecycle
stateDiagram-v2
[*] --> Idle
Idle --> Listening : user_spoke / voice detected
Listening --> Thinking : 1.2s delay
Thinking --> Streaming : first token received
Streaming --> Speaking : sentence_ready
Speaking --> Streaming : next sentence
Streaming --> Idle : [SILENCE] / end of response
Speaking --> Idle : finished_speaking + linger
Idle --> Proactive : autonomy trigger
Proactive --> Thinking : context available
Proactive --> Idle : no context / silence
Project Structure
B/
├── main.py # Entry point : boot sequence
├── core/
│ └── bus.py # EventBus : central nervous system
├── brain/
│ ├── llm.py # CognitiveEngine : LLM inference
│ ├── soul.py # StateMachine : emotions & stream buffer
│ ├── autonomy_loop.py # AutonomyEngine : proactive thought
│ ├── context.py # Context management
│ └── work_mode.py # Work mode prompt templates
├── vision/
│ ├── semantic.py # SemanticSensor : UIA extraction
│ └── mss_capture.py # VisionSensor : OCR fallback
├── sensors/
│ └── window_tracker.py # WindowTracker : focus detection
├── audio/
│ ├── speaker.py # VoiceEngine : TTS + DSP
│ └── ears.py # EarsSensor : STT
├── physics/
│ └── kinematics.py # KinematicsEngine : movement
├── ui/
│ ├── overlay.py # WindowManager : transparent overlay
│ ├── face.py # FaceRenderer : 60fps face
│ ├── chat.py # ChatBubble : text overlay
│ ├── input_box.py # InputBox : text input
│ ├── expressions.py # Expression definitions
│ └── theme.py # Visual theming
├── models/ # Local GGUF models (gitignored)
├── voices/ # Piper ONNX voice models
├── scripts/ # Setup & download utilities
├── docs/
│ ├── ARCHITECTURE.md # Detailed architecture docs
│ └── assets/ # Images & diagrams
├── .env # API keys (gitignored)
└── requirements.txt # Python dependencies
Core Systems
| System | Technology | Description |
|---|---|---|
| Cognitive Engine | Groq API / llama-cpp |
Cloud or local LLM inference for private, high-speed reasoning. |
| Semantic Vision | UIAutomation + Tesseract OCR |
B understands the context of your active windows and screen content. |
| Neural Hearing | Faster-Whisper |
Industry-grade transcription with neural VAD for reliable ears. |
| Vocal Synthesis | Piper TTS + Pedalboard DSP |
Low-latency, natural-sounding voice with robotic modulation effects. |
| Kinematics | PyQt6 QPropertyAnimation |
Smooth, 60fps movement with Bezier path interpolation and easing curves. |
| Event Bus | PyQt6 pyqtSignal |
Thread-safe pub/sub message broker : the central nervous system. |
Agentic Work Mode
Activated via Ctrl+Shift+Alt+W, Work Mode shifts B into a high-utility state:
- Semantic Monitoring : B monitors your progress on tasks in real-time.
- Contextual Curiosity : Proactively offers insights, documentation, or suggestions based on your current focus.
- Minimalist Presence : Dims facial expressions to minimize distraction while remaining vigilant.
flowchart TB
A["User presses Ctrl+Shift+Alt+W"] --> B["B asks: what are you working on?"]
B --> C["User defines goal"]
C --> D["B monitors screen context"]
D --> E{"Relevant content?"}
E -- "Yes" --> F["B offers insight / help"]
E -- "No" --> G["B stays silent"]
F --> D
G --> D
Hotkeys
| Shortcut | Action |
|---|---|
Ctrl+Shift+Alt+Q |
Kill switch : immediately terminates B and releases all system hooks. |
Ctrl+Shift+Alt+B |
Toggle input box : type messages to B. |
Ctrl+Shift+Alt+V |
Toggle speak mode : talk to B via microphone. |
Ctrl+Shift+Alt+W |
Toggle work mode : B becomes a proactive assistant. |
Installation
-
Clone the repository:
git clone https://github.com/Ahmad-Hassan-0/B---desktop-companion.git cd B---desktop-companion -
Set up the environment:
python -m venv venv venv\Scripts\activate # Windows source venv/bin/activate # Linux/macOS pip install -r requirements.txt
-
Configure API keys:
cp .env.example .env # Edit .env with your Groq API key (get one at https://console.groq.com/) -
Awaken B:
python main.py
Safety
Caution
Global Kill Switch: Ctrl+Shift+Alt+Q
This hotkey immediately terminates B and releases all system hooks. Use this if B becomes over-eager or if you need an instant exit.
Because B lives on a transparent, click-through overlay without a standard close button, the kill switch is the only way to exit. It is registered at the Win32 level and works even if the Qt event loop is unresponsive.
Technical Constraints
| Constraint | Target |
|---|---|
| CPU | Intel i5 (Quad-Core) or equivalent |
| RAM | 16 GB |
| Display | Any resolution (adaptive) |
| OS | Windows 10/11 (primary), Linux/macOS (experimental) |
| Tick Rate | 60 fps (16ms interval) |
| Inference | Groq API (cloud) or llama-cpp (local, 4GB+ model) |
Built by Ahmad Hassan
Wiring the soul, one tick at a time.