Hermes Agent keeps every conversation, tool result and reasoning trace in a single SQLite database, the state database (state.db), so that past sessions stay searchable across CLI and gateway runs. That search layer is FTS5 — and until recently, it was storing the same bytes three times over.
The old layout kept two full-text indexes (a Porter-stemmer index and a trigram index for CJK substring search), and both were inline: each one kept its own private copy of every message's content. The message body existed once in the messages table, then again in each index. On an agent that runs tools all day, tool output dominates the bytes — and it was being copied into all three places.
A real-world breakdown from the upstream issue tells the story: of a 505 MB database, 358 MB — 71% — was the FTS indexes alone, with the trigram index at 247 MB. The actual conversation data was under 30% of the file.
The fix landed as the compact v23 FTS layout. It changes two things:
- External-content indexes. The indexes no longer store their own copy of the content; they point at the real columns in the messages table. The duplication disappears.
- Tool rows leave the trigram index. CJK substring search inside tool output (base64 blobs, file dumps) is the rarest query there is, and it was the most expensive thing to index. It now falls back to a plain LIKE scan.
The migration is opt-in for existing installs — one command:
hermes sessions optimize-storageIt checks disk headroom first, demotes the legacy indexes, backfills the new ones in chunks (keeping the write-lock duty cycle low so a live gateway stays responsive), tears down the old shadow tables, and finishes with a VACUUM. It is safe to interrupt and re-run, and the messages table stays byte-identical — an independent review of a real migration confirmed matching SHA-256 fingerprints before and after.
Our numbers
I ran it on an installation with 40,835 indexed entries. The result:
Database size: 513.1 MB -> 217.3 MB (reclaimed 295.8 MB)That is a 58% reduction — right in the middle of the 60–78% range reported upstream, which varies with how tool-heavy the workload is. The whole run took a couple of minutes in the foreground, search worked unchanged immediately after, and no conversation history was touched.
Two practical notes from the run:
- The command asks for a y/N confirmation, so under automation pipe the answer in:
yes | hermes sessions optimize-storage. - This is a one-time layout migration, not a recurring chore. Fresh installs are born on the new layout; Hermes also runs its own automatic prune-and-VACUUM maintenance at startup. Going forward the database grows much slower — there is simply no second and third copy of every byte anymore.
If you have been running an agent for months and the database has quietly grown to half a gigabyte, the fix is one command and about 300 MB of your disk back.
