Note: this project has been a practice in not being so perfectionist, and the documentation in particular is in need of cleanup.

Welcome to my "tinker cast" - a personal project. It is at its core an extension of the computer science actor model with "personal knowledge management" (PKM) through plaintext Markdown notes. I chose Markdown over JSON, a database, or other options because I can use Obsidian or any other app for reading and writing my text notes, allowing them to act as a shared memory between me and my cast of actors.

Built on top of that is a process for automatic transcription of captured voice notes, along with various other input sources.

I've wrapped Akka's actor model implementation with a few specific additions

• my actors can easily use Markdown notes for memory - inspired from Obsidian's "vault"
• a markdown-based notification center is available to actors
• a "tinkerbrain" visualization, like so

Click here for a more historical Imgur album.

The initial problem solved was to replace Google Keep and Google Pixel Recorder, two apps I was using for transcribed voice notes. When OpenAI released Whisper, I replaced those Google apps with a fully local solution using Easy Voice Recorder and Syncthing - an actor watches for new files, and then they get transcribed and added to my Obsidian vault.

An important application of voice notes for me was cat litter tracking. One of my cats has a life-threatening chronic condition, and tracking his litter use gives me peace of mind in good times and helps at the vet in bad times. The generated summary reports look like:

LLMs are not used for this flow, though some light machine learning is, so when events are ambiguous, they go to an inbox instead so that I can manually fix the note.

When I started this project:

• I had recently left a hybrid backend and data engineering role, where I worked with a former Evernote CTO
• One of my two cats had recently developed a life-threatening chronic condition
• I had started a personal wiki of networked notes ("PKMS" = personal knowledge management system)

This project represents the intersection of those things. The name is a play on words, with "Cast" referring to the underlying actor model from computer science, as well as an allusion to casting magic spells (I does feel like magic sometimes when I change my lights with my voice).

So, I wanted to use my data engineering knowledge to automate the process of leveraging my personal wiki in order to help me take care of my cats. The nodes (circles) in the graphic below show my actors/atomic agents and the edges (lines) indicate their communication.

• (older graphic)
• click here for an animation

Here, we see atomic agents dedicated to

• My cats 🐱 (in brown)
• My smart lights 🕹️💡
• A sleep report 😴 and my Fitbit ⌚️
• Tracking when I last ate and reminding me if needed 🫢🥗
• Enabling other agents to react to time passing ⏰
• A notification center ❗️
  • Notifications via Ntfy 📧
• HALT and CO2 tracking 🛑
• etc

• Tech stack includes
  • Akka 2.6's Behaviors DSL for the actor model (Scala)
  • OpenAI Whisper for transcription
  • Rasa open source for entity extraction
  • Python/Flask for surfacing Whisper, Rasa, and other libraries
• The actors/atomic agents are generally explicit encodings in Scala, rather than AI
• Most of it works fully offline
• The visualization is a hack, frontend is my biggest weakness

• Knowledge management (links)
  • A text renaissance (yay for explicit encodings)
  • Networked thinking / atomic notes
  • Mind gardening
• The Extended Mind (2021 book)
  • You can almost think of this as part 4, "Thinking With Code"
  • (...and AI if/when needed)
• Kaizen
• Kasparov's Law

  • [...] which states that a human of average intelligence and an AI system working together in harmony is more effective than either working alone, and even more advantageous than a brilliant human working with a system poorly

  • Chess shows that humans and AI work better together (irishexaminer)

    • The ability to work efficiently with AI matters more for cyborg chess players than their standalone strength in chess. Several cyborg chess competitions were not won by the strongest attending chess players. [...]

  • I personally don't see AI as integral to Kasparov's law, it applies to tech in general
  • As a specific example, my cat litter tracking was initially supposed to answer the question, "When did Peanut last pee?" but I built the summary report along the way, and that (combined with infrequent manual investigations) has probably been a better trade-off for now

• "Cognitive light cones" by Michael Levin
  • You can learn more on Youtube or through his paper
  • Related to the similarly-named idea from physics, our cognitive light cones include the potential inputs to our current moment
  • My Tinker Cast helps provide the right past information at the right moment to inform future decisions
• The Thousand Brains hypothesis
  • You can think of the actors as externalized reference frames
  • However many cortical columns were dedicated to worrying about before, I got them back; it's kind of like having expanded my cortical column count by adding specialized virtual ones
  • I would say that I'm "Building a Thousand Brains" more than "Building a Second Brain" (a productivity brand)
  • I need to follow Monty's example and add functionality for anticipation/suprises and voting / Bayes' theorem

It can be thought of a civilian version of the "hyper-enabled operator"

• AI-Powered Super Soldiers Are More Than Just a Pipe Dream (Wired, may be paywalled)
• The Hyper-Enabled Operator (Small Wars Journal linked to by Wired above, archived link)

  • USSOCOM defines HEO as a SOF [special operation forces] professional empowered by technologies that enhance the operator’s cognition at the edge by increasing situational awareness, reducing cognitive load, and accelerating decision making.  In short, HEO hyper-enables the operator by providing technological aids to shorten the time within his OODA [observe, orient, decide, act] loop, thereby providing him with cognitive overmatch.

  • I have my situational awareness around: my cats, my air quality, and HALT-related things
  • I have less cognitive load, more peace of mind, fewer context switches
  • I can make faster decisions because data has already been gathered and digested
  • If applied to video games (say, Age of Empires or DOTA2) I might achieve "cognitive overmatch"

• The Raven Paradox
• Inventor of the world wide web wants us to reclaim our data from tech giants
• Eagleman Ted talk - creating new senses for humans
• Bret Victor - Inventing on Principle
• Luck Surface Area
  • Luck = [Doing Things] * [Telling People]
• Spoon Theory
• "power armor without the armor" and "pattern specialist"

Currently listening to: How to Measure Anything

Similar projects

• Atomic Agents (Python, available on pypi)
• Jido (Elixir)