B is not just an AI : he is a digital lifeform designed to live on your desktop. Built with a soul-first architecture, B observes your workflow, listens with neural precision, and interacts through a high-performance, glassmorphic overlay.
Explore the Vision • Architecture • Getting Started • Contributing • Privacy
Traditional assistants wait for a command. B waits for a moment.
B is designed as a proactive desktop companion. Using a 60fps event-driven central nervous system, B synchronizes his emotional state with your environment. He sees your screen semantically, tracks your active focus, and intervenes only when he has something truly valuable to contribute.
- Silent by Default : B respects your deep work state.
- Emotionally Aware : A complex internal state machine drives expressions and curiosity.
- Agentic Autonomy : B doesn't just respond; he thinks, wonders, and observes.
B is built on a centralized asynchronous pub/sub event bus : the EventBus. Every module communicates exclusively through this bus. No module knows about any other module. This strict decoupling makes the system testable, maintainable, and resilient.
flowchart TB
subgraph P["Perception Layer"]
WT["WindowTracker"]
SS["SemanticSensor"]
VS["VisionSensor OCR"]
end
subgraph C["Cognitive Layer"]
CE["CognitiveEngine"]
SM["StateMachine"]
AE["AutonomyEngine"]
end
subgraph O["Output Layer"]
VE["VoiceEngine"]
FR["FaceRenderer"]
CB["ChatBubble"]
KE["KinematicsEngine"]
end
subgraph I["Core Infrastructure"]
EB["EventBus"]
end
WT -- "active_window_changed" --> EB
SS -- "context_updated" --> EB
VS -- "context_updated" --> EB
EB -- "context_updated" --> CE
EB -- "tick" --> SM
EB -- "tick" --> KE
CE -- "b_spoke" --> EB
EB -- "b_spoke" --> VE
EB -- "b_spoke" --> FR
EB -- "b_spoke" --> CB
CE -- "b_move_request" --> EB
EB -- "b_move_request" --> KE
AE -- "trigger_proactive_thought" --> EB
EB -- "trigger_proactive_thought" --> CE
The data flows through B in a deterministic pipeline: Perception → Cognition → Expression.
sequenceDiagram
participant WT as WindowTracker
participant SS as SemanticSensor
participant VS as VisionSensor
participant CE as CognitiveEngine
participant VE as VoiceEngine
participant FR as FaceRenderer
WT->>SS: active_window_changed
activate SS
SS->>SS: UIA Tree Walk
alt Extraction Success
SS->>CE: context_updated
else Failure / Cooldown
SS->>VS: semantic_extraction_failed
VS->>VS: OCR Capture
VS->>CE: context_updated
end
deactivate SS
CE->>CE: LLM Inference
CE->>VE: b_spoke (Text + Emotion)
activate VE
VE->>VE: TTS + DSP Vocoder
VE->>FR: speaking_start
VE->>VE: Audio Output
VE->>FR: speaking_end
deactivate VE
CE->>FR: b_move_request (Spatial)
| Module | File | Responsibility |
|---|---|---|
| EventBus | core/bus.py |
Thread-safe pub/sub message broker. All inter-module communication flows through this. |
| main.py | main.py |
Boot sequence : instantiates all modules, starts the 60fps tick timer, registers global hotkeys. |
| Module | File | Responsibility |
|---|---|---|
| WindowTracker | sensors/window_tracker.py |
Hook-based active window change detection. Fires when the user switches focus. |
| SemanticSensor | vision/semantic.py |
UIA-based DOM/window tree walking. Extracts structured content with quality scoring and adaptive cooldown. |
| VisionSensor | vision/mss_capture.py |
OCR fallback pipeline using MSS + Tesseract for frameworks incompatible with UIA. |
| Module | File | Responsibility |
|---|---|---|
| CognitiveEngine | brain/llm.py |
LLM inference orchestration (Groq cloud or local llama-cpp). Manages context, history, spatial mapping, and streaming token parsing. |
| StateMachine | brain/soul.py |
B's emotional state : blinking, resting, conversing. Real-time stream buffer that parses LLM output into sentences. |
| AutonomyEngine | brain/autonomy_loop.py |
Proactive thought scheduling : decides when B should speak unprompted based on context quality and timing. |
| Module | File | Responsibility |
|---|---|---|
| VoiceEngine | audio/speaker.py |
Piper ONNX TTS with DSP vocoder chain (pitch shift, bitcrush, chorus) for robotic modulation. |
| FaceRenderer | ui/face.py |
PyQt6 QPainter-based hardware-accelerated face rendering at 60fps. |
| ChatBubble | ui/chat.py |
Glassmorphic chat overlay that displays B's spoken text. |
| KinematicsEngine | physics/kinematics.py |
Physics-based movement with Bezier path interpolation and easing curves. |
| EarsSensor | audio/ears.py |
Speech-to-text via Faster-Whisper with neural VAD. |
flowchart LR
subgraph Input["Input"]
A["User Types"]
B["User Speaks"]
C["Screen Changes"]
end
subgraph Process["Processing"]
D["EventBus"]
E["CognitiveEngine"]
F["StateMachine"]
end
subgraph Output["Output"]
G["FaceRenderer"]
H["VoiceEngine"]
I["ChatBubble"]
J["Kinematics"]
end
A -- types --> D
B -- speaks --> D
C -- changes --> D
D -- routes --> E
E -- drives --> F
F -- animates --> G
F -- speaks --> H
F -- displays --> I
E -- moves --> J
stateDiagram-v2
[*] --> Idle
Idle --> Listening : user_spoke / voice detected
Listening --> Thinking : 1.2s delay
Thinking --> Streaming : first token received
Streaming --> Speaking : sentence_ready
Speaking --> Streaming : next sentence
Streaming --> Idle : [SILENCE] / end of response
Speaking --> Idle : finished_speaking + linger
Idle --> Proactive : autonomy trigger
Proactive --> Thinking : context available
Proactive --> Idle : no context / silence
B/
├── main.py # Entry point : boot sequence
├── core/
│ └── bus.py # EventBus : central nervous system
├── brain/
│ ├── llm.py # CognitiveEngine : LLM inference
│ ├── soul.py # StateMachine : emotions & stream buffer
│ ├── autonomy_loop.py # AutonomyEngine : proactive thought
│ ├── context.py # Context management
│ └── work_mode.py # Work mode prompt templates
├── vision/
│ ├── semantic.py # SemanticSensor : UIA extraction
│ └── mss_capture.py # VisionSensor : OCR fallback
├── sensors/
│ └── window_tracker.py # WindowTracker : focus detection
├── audio/
│ ├── speaker.py # VoiceEngine : TTS + DSP
│ └── ears.py # EarsSensor : STT
├── physics/
│ └── kinematics.py # KinematicsEngine : movement
├── ui/
│ ├── overlay.py # WindowManager : transparent overlay
│ ├── face.py # FaceRenderer : 60fps face
│ ├── chat.py # ChatBubble : text overlay
│ ├── input_box.py # InputBox : text input
│ ├── expressions.py # Expression definitions
│ └── theme.py # Visual theming
├── models/ # Local GGUF models (gitignored)
├── voices/ # Piper ONNX voice models
├── scripts/ # Setup & download utilities
├── docs/
│ ├── ARCHITECTURE.md # Detailed architecture docs
│ └── assets/ # Images & diagrams
├── .env # API keys (gitignored)
└── requirements.txt # Python dependencies
| System | Technology | Description |
|---|---|---|
| Cognitive Engine | Groq API / llama-cpp |
Cloud or local LLM inference for private, high-speed reasoning. |
| Semantic Vision | UIAutomation + Tesseract OCR |
B understands the context of your active windows and screen content. |
| Neural Hearing | Faster-Whisper |
Industry-grade transcription with neural VAD for reliable ears. |
| Vocal Synthesis | Piper TTS + Pedalboard DSP |
Low-latency, natural-sounding voice with robotic modulation effects. |
| Kinematics | PyQt6 QPropertyAnimation |
Smooth, 60fps movement with Bezier path interpolation and easing curves. |
| Event Bus | PyQt6 pyqtSignal |
Thread-safe pub/sub message broker : the central nervous system. |
Activated via Ctrl+Shift+Alt+W, Work Mode shifts B into a high-utility state:
- Semantic Monitoring : B monitors your progress on tasks in real-time.
- Contextual Curiosity : Proactively offers insights, documentation, or suggestions based on your current focus.
- Minimalist Presence : Dims facial expressions to minimize distraction while remaining vigilant.
flowchart TB
A["User presses Ctrl+Shift+Alt+W"] --> B["B asks: what are you working on?"]
B --> C["User defines goal"]
C --> D["B monitors screen context"]
D --> E{"Relevant content?"}
E -- "Yes" --> F["B offers insight / help"]
E -- "No" --> G["B stays silent"]
F --> D
G --> D
| Shortcut | Action |
|---|---|
Ctrl+Shift+Alt+Q |
Kill switch : immediately terminates B and releases all system hooks. |
Ctrl+Shift+Alt+B |
Toggle input box : type messages to B. |
Ctrl+Shift+Alt+V |
Toggle speak mode : talk to B via microphone. |
Ctrl+Shift+Alt+W |
Toggle work mode : B becomes a proactive assistant. |
-
Clone the repository:
git clone https://github.com/Ahmad-Hassan-0/B---desktop-companion.git cd B---desktop-companion -
Set up the environment:
python -m venv venv venv\Scripts\activate # Windows source venv/bin/activate # Linux/macOS pip install -r requirements.txt
-
Configure API keys:
cp .env.example .env # Edit .env with your Groq API key (get one at https://console.groq.com/) -
Awaken B:
python main.py
Caution
Global Kill Switch: Ctrl+Shift+Alt+Q
This hotkey immediately terminates B and releases all system hooks. Use this if B becomes over-eager or if you need an instant exit.
Because B lives on a transparent, click-through overlay without a standard close button, the kill switch is the only way to exit. It is registered at the Win32 level and works even if the Qt event loop is unresponsive.
| Constraint | Target |
|---|---|
| CPU | Intel i5 (Quad-Core) or equivalent |
| RAM | 16 GB |
| Display | Any resolution (adaptive) |
| OS | Windows 10/11 (primary), Linux/macOS (experimental) |
| Tick Rate | 60 fps (16ms interval) |
| Inference | Groq API (cloud) or llama-cpp (local, 4GB+ model) |
Built by Ahmad Hassan
Wiring the soul, one tick at a time.
