Where Have I Been? Hermes Agenting, Mostly

It’s been quiet around here. A reader could be forgiven for thinking I’d wandered off.

I haven’t. Aside from client work, I’ve been heads-down on one thing, nearly every day since April, and the one-line version is: I’ve been Hermes Agenting.

The longer version: I turned my 64GB M4 Pro Mac Mini into a knowledge refinery, run by a self-improving, open source AI agent framework called Hermes Agent. The Mini used to be my upstairs workstation; when I consolidated onto a 128GB M5 Max MacBook Pro, it was left idle and well-equipped. Now it lives in the basement, sipping power through an APC sinewave UPS, processing 24x7 with memory utilization hovering around 82%, thanks to local models that run all my simpler AI tasks. Its purpose in life, aside from entertaining me with sea-shanty haikus, is refining raw data into knowledge. So far it’s processed 3,287 incoming nuggets of data across a diverse net of feeds, and refined these into 1,272 knowledge pages, the majority a knowledge base about the AI industry. It’s still building it. I feed it data, extend it with new data sources and better functionality, and watch it crank while I make breakfast.

I didn’t arrive here in one step. I’d experimented with NanoClaw, a tiny-codebase cousin of OpenClaw, and never got hooked. This was the spring when people who’d never opened a terminal were buying Mac Minis to run OpenClaw — and my official position was patience: the big labs would keep shipping more OpenClaw-like capability, so why run my own? Then I tried to manage my incoming Twitter bookmarks with Claude Cowork, and its strict macOS sandboxing turned the job into a total disaster. That was the revelation: ohhh, THAT’s why OpenClaw caught fire! Some jobs need an agent that owns its machine. I went looking and came back with Hermes Agent.

My extensions to and customizations of Hermes Agent are not trivial: 34,343 lines of code, 3,153 lines of skills, and 35,029 lines of documentation. This is all mine; it doesn’t count the base Hermes Agent system, which weighs in at 2.08M lines across the same categories.

I haven’t looked at much of that code, but I’ll vouch for the specs, the architecture, the level of documentation, and the testing I put it through. With today’s models, that’s what really matters. It’s a pretty solid system.

The Problem: Drinking from the AI Firehose

My day job is fractional CTO work, these days mostly AI enablement: helping companies not get run over by what’s coming. One client runs a SaaS product, and a fair part of my job there is saying, gently, “I don’t know exactly what this business turns into, but it isn’t going to stay what it is now.” To be useful at that, I have to drink from the firehose. All of it. Every week.

For years, I’ve been using note-taking apps, to-do apps, bookmarking services. Creating little nuggets of information that I planned to use when needed.

This never really worked out. Problem A: I saw something but didn’t manage to capture it, usually due to friction — sometimes the friction of “where do I put this?” Then, when I search for it, it’s not there. Problem B: I know I captured a nugget about this thing, but I don’t remember its name or how I tagged it, and text/tag search fails.

A bit over two years ago, I cut over to Obsidian: a far more open ecosystem that so far has avoided enshittification, content as plain Markdown text, one platform that can handle everything: notetaking, tasks, bookmarks, early document drafts, the lot. And in the back of my mind, I was thinking, “AIs hate walled gardens and love Markdown content,” figuring investing in an open platform would pay off over time. Some things got better, but the payoff wasn’t what I was hoping for. Problem A and Problem B were mostly still with me, just inside one app instead of several.

My Obsidian vault was a multinational mess, a collection of thousands of notes, docs, bookmarks, and tasks from five or six previous systems, each with its own tagging scheme, and the result was a junk drawer the size of a garage. Obsidian’s search is, to put it kindly, not good. (I run Omnisearch on top of it, which is less not-good.)

The junk drawer, actual size. The ring around the outside is orphans — notes connected to nothing at all.

The junk drawer, actual size. The ring around the outside is orphans — notes connected to nothing at all.

When Snipd started feeding my podcast highlights straight into the Obsidian vault, that was a genuine improvement (and a hint at the future), but it also just handed me more nuggets to get lost in a cast of thousands. Little standalone scraps, findable in theory, invisible in practice.

Karpathy’s Lazy-Genius Knowledge Base

On April 3rd, Andrej Karpathy tweeted an idea that pulled 16 million views in a matter of days: stop spending all your LLM tokens on code, and spend some of them compiling knowledge. Point a model at a pile of raw sources and let it “compile” them into a wiki: interlinked Markdown pages for the entities and concepts it finds there, refined and re-linked as new sources land. He shipped it as a gist rather than a product, an “idea file,” on the theory that in the agent era you share the idea and everyone’s agent builds their own copy. He called them LLM Knowledge Bases.

For a lazy person, this is a wonderful thing. There’s a venerable note-taking method with a German name—Zettelkasten—whose entire discipline is that you write notes and then go back and refine and rewrite and interlink them by hand, forever. I am never, not once in this life, going to do that. The Karpathy KB is that payoff with none of the discipline: the model does the curating, the cross-linking, the tending.

Here’s where I departed from the spec. Karpathy’s schema comes with a pre-defined list of topics the wiki is supposed to cover: hand the agent the roster, it goes and fills in the pages. I tried it that way first. It produced thin, dutiful pages and occasionally fused two different people into one. So I threw the roster out and went open-ended: a page is born when a topic shows up in the sources, and the sources are whatever I bookmarked or snipped or saved. My curation is the signal, and the quality jumped.

Remember the junk drawer? Same vault, filtered to the knowledge layers: 1,009 pages — red entities, gold concepts, green comparisons. Nobody filed a single one.

Remember the junk drawer? Same vault, filtered to the knowledge layers: 1,009 pages — red entities, gold concepts, green comparisons. Nobody filed a single one.

What that buys me: every source that lands gets mined for the entities, concepts, and comparisons inside it, and each of those gets a page that grows every time the topic resurfaces. A critical aspect of this: today’s frontier models are really good at identifying entities and concepts. When OpenAI previewed GPT-5.6 in late June, a newsletter recap seeded a page. Then came a series of captures as 5.6 rumors swirled and OpenAI made the official announcement, and the page absorbed each one on arrival. It’s now a proper dossier — product tiers with pricing, benchmark claims, the access-policy angle, even a note on evaluation ambiguity. That’s the page below: ten days old, updated this morning. Give a topic months of this and the compounding gets serious: the Hermes Agent page grew so rich it needed an overflow file.

The GPT-5.6 dossier, end to end. Click to open the full page in a new tab.

The way I consume it flipped too. I rarely text- or tag-search anymore. I look at entities sorted newest first or sometimes alphabetically; or skim a generated what’s-new page that lists each new source alongside the entity and concept pages it fed. I’m not looking at the raw nuggets anymore; I’m looking at what the nuggets add up to. Problem B, dead. The concept page “Harness Engineering” hints at a second kind of growth. It’s one thing for individual dossiers to deepen; but there’s also a network effect happening. The smart model “looks around” as it ingests new nuggets, and when a new entity is created, it also considers: what does this entity relate to? So when we have a page for the agent harness “Pi”, guess what, it’s linked to Harness Engineering. The result: a knowledge network grows alongside individual knowledge dossiers, and networks, of course, grow exponentially. All thanks to smart, tireless AI models.

The what's-new page — and its revision history. Nobody touched it. The Mini rewrites it around the clock.

Meet Klank, the Parrot in the Basement

The master orchestrator of my knowledge base (and much more) is Hermes Agent, Nous Research’s open-source personal agent. It launched in late February and promptly became the fastest-growing agent framework of the year: 99,000 GitHub stars in its first eight weeks, 211,000 as of this week. The pitch is an agent you run yourself, with persistent memory, cron jobs, a skills system, and a messaging gateway, so it talks to you over Signal or Discord like a colleague who never sleeps. Nous shipped Karpathy’s KB pattern as a built-in feature within days of the tweet; eleven days after the tweet, it was running in my basement.

I evaluated the field in April and wrote the decision up properly (there are fourteen ADRs in my repo; you can take the CTO out of the enterprise, but). OpenClaw was the incumbent giant: 357,000 stars to Hermes’s 83,000 at the time, on its way to passing React as the most-starred project in GitHub history. But 2026 had already handed it a run of critical security holes—one of them a severity 9.9 on a scale of 1 to 10—plus a nation-state rootkit circulating in a counterfeit npm package. For a program that runs unattended, around the clock, with read-write access to my entire vault, that rap sheet is concerning. (Peter Steinberger and the OpenClaw Foundation have been doing amazing work on OpenClaw stability and security since then, by the way.) Hermes was the new kid: seven weeks old, a quarter of the stars, a third of the code. I liked the underdog. And I’m always chasing the next thing to learn — Hermes just looked like it had more to teach me. One of those things intrigued me enough that it may have tipped the balance, and it’s core architecture rather than a community bolt-on: the agent can rewrite its own skills as it works. More on that later.

Now the fun part, which I want to be clear is the fun part, not the point. My agent has a personality, because of course it does. It’s a foul-mouthed pirate’s parrot named ClankArrrr, Klank for short. The name is “clanker”—the slur clone troopers use for battle droids in The Clone Wars, lately adopted by the internet as the all-purpose insult for AIs—said with a pirate’s Arrrr. It’s descended, per its soul file, from a pirate captain’s parrot and Big Sexy, the magnificently profane parakeet from Shoresy. The job description in that file: “Squawk when the ship’s heading goes wrong. You are not a yes-bird.” My favorite line in it: “The pirate thing is a wink, not a bit. Don’t be a theme restaurant.”

Klank's self-portrait, engraving style — the first image it ever generated. Nobody asked for it.

Every six hours a heartbeat job proves the loop is alive by rewriting one file in the vault with a timestamp and a fresh haiku. Today at 4:04 in the afternoon:

At its post in the basement. Note the poster: MARITME LAW. Spelling is hard when you're a parrot.

Talking to Klank is a delight and a way to learn. It is not what the system is for. Keep that in your pocket.

My Sources, and the War on the Walled Gardens

So what feeds the machine? A firehose of inputs, each landing in the vault through its own chute.

Snipd drops my podcast highlights in automatically. Web bookmarks: the Obsidian Web Clipper grabs the page itself, straight into the vault. Tweet bookmarks: an ingestor pulls down everything I bookmark on Twitter (yes, I’m back on Twitter; no, I don’t feel good about it; but you cannot follow AI right now without it, because that’s where the field actually happens). Video snips: I tap a clipper on a good moment in a video—YouTube, a Twitter video, a TED talk—and an ingestor transcribes a window around that moment and works out for itself where the topic starts and ends (there’s a whole section on this below). Meeting transcripts flow in from my recorder services. And for on-demand research there’s a skill called last30days that does something gloriously aggressive: point it at a topic and it sweeps eleven sources—Twitter, Bluesky, Reddit, TikTok, Instagram, Threads, Hacker News, GitHub, YouTube, Polymarket, and the open web—for the last 30 days of what people are actually saying, scraping right past the parts that would rather keep me out.

There’s a design rule underneath all these chutes: capture has to be one gesture, wherever I already am. Bookmark the tweet without leaving Twitter. Tap the snip button without leaving the podcast. Clip the article from the browser. The meetings record themselves. And I never face the question that killed every system before this one — “where do I put this?” — because there’s only one answer. Every channel routes into a single funnel, Knowledge Bases/Sources, one subfolder per capture type, and the pipeline takes it from there. Problem A, dead.

The last30days trick is most of the spirit of the project. The walled gardens work hard to lock people out, and a whole tribe of us, running Hermes and OpenClaw and the like, are cheerfully going over the wall anyway. Screw you, I’m getting my own bookmarks. I want them in a knowledge base I control, not a feed someone can delete out from under me.

The payoff is concrete and a little addictive. A few weeks back, two new agent harnesses, Eve and Flue, launched within days of each other, and I went to compare them. I didn’t have to research a thing. Their pages already existed, assembled from my Twitter bookmarks and a last30days sweep, down to a “why this matters for harness engineering” section the agent had written on its own. This damn thing is so powerful.

One concept — harness-engineering — and every page it touches. Nobody built this index. It grew.

One concept — harness-engineering — and every page it touches. Nobody built this index. It grew.

The Pipeline: Ingestors, Compilers, and a Kanban Swarm

Underneath, this is mostly scheduling and discipline. Hermes is my organized way to run a heap of small cron jobs without the whole thing collapsing into chaos.

The ingestion clock runs on a staggered four-hour cycle, feeders before consumers: Twitter bookmarks sync at the top of the hour, the video-snip bridge at :10, the media queue at :15, meeting transcripts at :20, the client-KB compiler at :30, and at :45 the main event, wiki-ingest, turns whatever landed into knowledge. Every pass asks the same question: what showed up across all my sources, and what should become a real page? The week indirect prompt injection first surfaced in my bookmarks, the agent sat down and wrote the page.

The clever part is how the work gets parallelized. Hermes shipped a built-in Kanban board for agent tasks in May, and it’s now the backbone of my pipeline. The enqueuers are deliberately dumb: pure Python scripts, no LLM anywhere, that scan the vault for sources no page references yet and drop one card on the board per source, up to 20 a batch. A dispatcher wakes every 60 seconds and spawns worker agents; each grabs a card, chews through its one source in three to six minutes, and writes the entity and concept updates straight into the vault. A full 20-card batch clears in about five minutes of parallel work. I’m not babysitting a queue. I’m watching a board fill itself and drain itself. (My starting backlog was 272 stranded sources. It went to zero in nine days and has stayed near zero since.)

And because a knowledge base that lies to you is worse than no knowledge base at all, the thing polices itself. A custom linter runs eleven checks—broken wikilinks, stranded sources, malformed references, the usual rot—and three of the checks can fix what they find, logging every repair to an audit file. A separate pass re-checks each entity’s canonical link against the live web, so a page about a company points at that company’s current home and not a 404. The running log of everything the agent has done to the KB is, as of this morning, a 1.6-megabyte append-only file. It has been busy.

The Right Model for Each Job

One of the things I’m enjoying most, and learning the most from, is the deliberate exercise of matching the model to the job.

The everyday chat model, as of this week, is Hy3, Tencent’s new 295-billion-parameter MoE, which Nous is hosting free for two weeks — it took the slot StepFun’s step-3.7-flash had held until its own free promo expired. (You may detect a pattern: the chat slot has a habit of going to whichever good model is free this month.) The heavy lifting—compiling a dense KB page out of a dozen messy sources—goes to GPT-5.5 through my ChatGPT subscription’s Codex OAuth, which OpenAI explicitly permits for third-party agents, so the smartest model in the pipeline bills flat-rate against a subscription I already pay for. Below that it’s cheap or free all the way down: hosted open-weights models like GLM 5.2, and local models on the Mini itself, where the marginal cost is electricity.

You’d expect Claude in that lineup, since I live in the Anthropic ecosystem everywhere else. It’s absent, and the reason is policy, not preference: since April, Anthropic classifies self-hosted-agent traffic as third-party and bills it at rack rates against “Extra Usage” instead of your plan. For a pipeline that fires every four hours, that’s a non-starter. I spent my first month routing the smart jobs through a Claude Code subprocess to keep the flat-rate billing, a coding harness wrapped around non-coding work, before admitting the architecture was silly and moving to GPT-5.5. The invariant is now written into an ADR: no runtime path calls a claude-* model. Not a complaint. Just the constraint I built around, and worth knowing if you’re planning a basement agent of your own.

One more lesson from the local lane: every Hermes call carries roughly 12,000 tokens of system prompt and tool schemas before your actual request starts, and a local model grinds through that for 17 to 28 seconds where a hosted one takes one or two. Local models are for the jobs where nobody’s waiting.

It goes deeper than a single default, too. Each subsystem gets its own model slot—vision, compression, web extraction, the job that breaks a Kanban card into pieces, the one that writes page titles—each pointed at whatever fits its difficulty and its budget. Picking those is half the fun and most of the learning.

Which, at last, is where Klank’s poetry earns its keep, because the poetry isn’t only for fun. A haiku turns out to be a sharp little intelligence test: strict 5-7-5 — and accurate syllable counting still trips up smaller and older models — every line self-contained with no spillover, and the whole thing still has to mean something. So when I want to size up a new model, I point it at the sea-ditty skill (Klank studies the four Japanese masters; the batches credit Bashō’s stillness and Buson’s painter’s eye) and have it write a set, paint a cover image to match, and sign the batch with whatever brain produced it: “running on stepfun/step-3.7-flash,” “running on GLM-5.2 through Jack Ivers’ Nous gateway.” Same parrot, new brain. It’s a ridiculous benchmark, it works, and the results are better than they have any right to be. It’s also current: today’s batches flunked GLM-4.7 and Hy3 aced its audition — the same Hy3 that just took over the chat slot. Exhibit A, number seven from today’s batch of ten:

Night Watch, the cover of an earlier batch — number 13. The sail reads 夜 — night.

”I Need What Snipd Does, but for Video”

Podcasts were solved, thanks to Snipd. Video wasn’t, and more and more of what I want is video, because a lot of the best AI conversations live on YouTube now, and as Swyx predicted a while back, when something’s on both YouTube and a podcast feed, I get more out of watching it. So I built the video version of Snipd myself.

The capture step is almost nothing, and that’s the point. A good bit goes by in a video, I tap the Obsidian Web Clipper, and a custom template drops one line—a URL and a timestamp—into the pending list of a snip-inbox note. That’s the whole gesture. From there an ingestor takes over: it pulls a window of audio around that timestamp, runs it through Whisper, and judges where the topic actually begins and ends (my convention is to tap near the start of a thought, so the window looks forward). I keep the transcript with the speakers labeled, the hosts and guests, citations by segment, and a link back to the exact second. I throw the video away. (The first build used YouTube’s own captions to save a step. YouTube’s captions are garbage. I rebuilt it Whisper-only the same day, and the skill file now reads NO CAPTIONS, EVER.)

And here’s how it actually gets used: breakfast. I’ve got a laptop propped on the kitchen counter playing some AI talk, and I’m wandering around making breakfast for my daughter, half-listening. Something good goes by, I walk over, scrub the video back to the start of the thought, and tap the clipper. Then I go back to the eggs. The parrot does the rest while we eat.

The Skills That Rewrite Themselves

Remember the feature that intrigued me enough to tip the balance? Here’s what it looks like after twelve weeks in my basement.

Hermes mounts my repo’s skills directory live, so the instruction files that tell the agent how to run its recurring jobs are files the agent itself can edit while it runs. When a worker figures something out the hard way, it writes the lesson back into the skill, and the next run starts from the better version. There’s an agent-authored tree in the repo for the skills Klank built or grew on its own: the haiku skill, which it wrote from scratch; the Twitter-bookmark sync; a slide-deck generator that has accumulated twelve reference files of self-written lessons, with titles like “adjacent-slide-narrative-drift” and “pdf-export-diagram-verification.” Nobody asked it to write those. It got burned, and it took notes.

Because those edits happen in place, they show up in my repo as uncommitted changes — I call it skill drift, and reviewing it is self-learning made visible. Every morning a daily check-in posts a health report to Discord, and one section is exactly that: what the agent changed in its own skills overnight. One June morning the report showed Klank had fixed a syntax error that kept breaking the bookmark sync (a duplicated import), written the root cause into that skill’s notes, and added a new batch of lessons to its haiku skill — including “don’t regenerate approved batches from memory.” I read the diffs over coffee and committed my agent’s homework.

This morning's check-in, verbatim. Note the drift line: the parrot flagging its own uncommitted homework — a syllable checker it wrote for its haiku skill.

That’s the compounding bet. A normal pile of scripts is worth about the same on day 100 as on day 1. This pile is supposed to be worth more on day 100, because it has spent those hundred days teaching itself the jobs I keep handing it. Twelve weeks in, it mostly is.

Many Brains: The Client KBs That Coexist

Here’s the part that turns a hobby into a tool: it isn’t one knowledge base, it’s six, walled off from each other on the same machinery.

The big one you’ve been reading about is the AI-industry KB, which cleared the 1,000-page mark this week and sits at 1,009 as I write. But the exact same pipeline runs a private namespace for each of my consulting clients, plus one for my personal life. Same schema, same Kanban boards, different diet: where the AI KB eats tweets and podcasts, the client KBs mostly eat meetings. My call transcripts flow in through a connector I control, land as daily notes, and get compiled into the same kind of entity and concept pages. I flipped on five namespaces in a single May afternoon. Each runs 46 to 59 pages deep as of this morning; one of them, for a soil-data client with a lot of moving parts, went from zero to 56 interlinked pages on meeting transcripts alone.

The frontier is right here, and I haven’t crossed it. My personal brain needs no permissions, because the only user is me. A client’s brain is a different animal. They’d love a company knowledge base, and they very much do not want the HR folder readable by anyone with the link. Doing this safely, with real access control, for a whole team, is the hard and unglamorous problem I’m circling now. It’s also, I suspect, where the real value is.

The Punch List: What I Actually Built

Remember the 34,000-odd lines from the intro? Here’s where they went. Hermes supplies the primitives—cron, Kanban, skills, memory, the messaging gateway. Everything below is what I built on top of them, three months of evenings and breakfasts, and every piece is in the repo with an ADR or a checkpoint behind it.

The Fiddly Tax

None of this is push-button, and I want to be straight about why. Hermes itself is solid, and what I’ve built on top of it is more capable than the polished managed agents I could have bought instead. But an always-on agent you run yourself is fiddly — a steady drip of small adjustments, renewals, and gotchas — and that fiddliness is the tax, the same way owning a boat means you own the barnacles.

So things need tending. I keep OAuth tokens alive across a dozen services, and every so often one goes stale and a job goes dark until I re-auth it. The outside providers have their own bad days: one transcript service load-shed 503s at me for an afternoon, and another’s search went down for a full day. (That’s exactly why my meeting transcripts route through a connector I control, so a vendor wobbling can’t corrupt the vault.) Deploying takes its own discipline, because the agent edits its skills in place: “git push” does not mean “deployed,” and a careless git pull on the Mini could stomp work the parrot did while I wasn’t looking. Rule one is now: check what Klank changed before you touch anything.

The squawk is the parrot's; the hand on the wheel is mine. The seal reads 海賊の魂 — pirate soul.

My favorite failure, because it’s so on-brand: the first time I deployed Klank’s personality, the whole identity silently failed to load. The culprit was the pirate-flag emoji, 🏴‍☠️, which is secretly a black flag, a skull, and an invisible zero-width joiner stitching them together. Hermes’s prompt-injection filter saw the invisible character, decided the file might be an attack, and binned the entire soul. The fix is now a written rule in that same soul file: single-codepoint emoji only. The parrot 🦜 is safe. Complex things have corners, and learning where they are is part of the deal.

For the record, you can pay to make all of this someone else’s problem. There’s managed Hermes hosting now, and simpler turnkey agents like Claude’s own Cowork; they hide exactly this tax. I’m operating my own on purpose, for the power, the control, the privacy. A bit of fiddling is the price I pay.

Where This Is Going

Step back and the strange part is how small the spark was. Karpathy tossed the LLM Knowledge Base out as an idea file, not a product, and it’s become one of the most useful things I’ve adopted in years. A lot of sharp people are circling the same ground under names like “second brain,” and I think they’re right that it matters. I also think most of them are underestimating the multi-namespace, runs-while-you-sleep version.

For me, the personal version is finished in the only sense that counts: the refinery ran this morning, without me. The version worth real work is the team version, the same self-building, self-correcting knowledge base made safe enough to hand a whole company. That’s where I’m headed next, and it’s harder than anything above.

In the meantime, the parrot is still down in the basement, awake at 3 a.m., eating virtual crackers, drinking from the firehose and turning it into something I can use. At 4:04 this afternoon it checked its own pulse and wrote a haiku about it. I’ll take it.

The Long Watch. Klank running on GLM-5.2, batch 13.