LLMs Aide in Developing a Pervasive Second Brain
I love many of the latest advancements in PKM. Tana, Mem, Obsidian, even Bardeem, and of course, Coda. The innovative approaches in keyboard expanders like Text Blaze and Espanso, also blend into the goals we all share concerning second brain activities.
What’s common about these tools? Their domains mostly end in .ai. There’s a correlation between modern second brain tools and AI. No surprise; AI might be the silver bullet that KM has needed for four decades.
What else is common about these tools? None of them play well together or operate in all the environments we tend to work where second brain activities occur.
The Interoperability Paradox
When I’m in Tana, I may stumble across something I saw in Slack that is particularly relevant about my current focus. I can go find it, or maybe I book-marked it in Mem. I recall bookmarking it, but where exactly?
This is a common and pervasive process; we see things that are sense-worthy; they are snippets of information that move us in some way, or compel us to take note. But, they happen across a vast horizon of tools and contexts. If you’re a new startup like Tana, you now need to build a few dozen API interfaces in an futile attempt to play everywhere.
This is the interoperability paradox; the number of connections and the effort required to build them is bigger than the product itself.
No single vendor of PKM can ever be everywhere. This is why glue-factories like Make and Zapier have thrived, despite serving as band-aide-like duct-tape, poorly architected Goldbergian shit shows, and deeply latent. These tools attract business logic (where it should not live) and create more security attack surfaces. Describing this as a shit show is probably being kind.
Note-Taking Funnel
I’ve often felt that capturing things that matter to us in the context of PKM, occurs everywhere. From the top of the information funnel all the way down deeply inside applications like a field in a database. The cone of sense-making activities is very wide and very deep. This illustration demonstrates how one tool that works well at the top of the funnel needs to interface with another tool forther down in the funnel.
Get Lucid
You might be sitting on a blog and realize there’s something related to a cell in a record in Airtable, which happens to be open in the adjacent browser tab. You’re determined to capture both of these observations, but your PKM platform of choice doesn’t work in either of them and it’s not likely to - ever.
What’s the Remedy?
I think the capacity for PKM interoperability intersects with two realities:
- The note-taking funnel is an abstraction; an economic externality to PKM tools.
- Artifacts gathered in this abstraction must be easily found when needed.
These are universal attributes that hold true in all PKM activities. From innovators to users, we generally have a lofty belief that a single PKM tool can meet many, if not all, interoperability and findability requirements.
If you own a really nice electric vehicle, it’s true value is determined by its ability to be conveniently charged. Elon Musk was smart; he realized infrastructure was a critical economic externality of Tesla’s product success.
Your second brain’s architecture is no different; it needs infrastructure for its value to be fully realized. The two realities above outline the infrastructure that’s missing. In my prototype, an oversimplification of it looks like this.
Early Assessment
This approach is untested and still very early. But it has proven a few noteworthy things that we can explore before I get deep into the implementation details and how this infrastructure works.
Interoperability
The diagram shows only a few app silos; however, this architecture works in all silos. The OS-level script makes it possible to homogenize PKM activities across all contexts from desktop apps to web browsers, and all without custom browser plugins. It even supports applications that have yet to be invented.
Findability
How do we make snippets that are captured in Coda (or any desktop app or browser for that matter), easily findable in AP Newswire? This is [seemingly] a search requirement. AI — specifically LLMs — lend themselves well to meeting this challenge.
Walkthrough
- OS-level scripts make it possible to capture information in any context
- LLMs provide word vectors; Pinecone provides vector storage and findability
- Cmd / → captures the current selection, vectorizes, and stores
- Cmd \ → captures the current selection, vectorizes, and recalls
Click to enlarge any of these example use case images.
Capturing Snippets
Coda Experience
- In Coda, highlight a noteworthy snippet
- Press Cmd /
- Title and keywords are automatically generated
- Curate (if necessary)
- Select to to capture
Recalling Snippets
Slack Experience
- In Slack, enter a topic
- Press Cmd /
- Browse the list
- Select to insert
Mem Experience
- In Mem, enter a topic
- Press Cmd \
- Browse the list
- Select to insert
Superhuman (Email) Experience
- In the email client, enter a topic in a new message
- Press Cmd \
- Browse the list
- Select to insert
FreeFlow
This article is a precursor to more tests and research concerning the use of this approach. I call it FreeFlow because that’s what I envisioned when trying to put a dent in the second brain interoperability paradox.
I simply want what I am compelled to take note of, to be findable (and useable) irrespective of tool context.
I want my sense-making snippets to flow freely amongst all apps; not just my PKM app. I want my PKM artifacts to be instantly (and smartly) recalled and used in every other computing context.
Tall order indeed.