AI & agents
A multi-agent system on own hardware — models, memory, jobs.
Many companies want AI agents without sending customer data to someone else’s API. This platform is my answer in my own operation: it runs on an NVIDIA DGX Spark (Grace Blackwell, 128 GB unified memory) plus a smaller second GPU box. vLLM serves open-weight models in NVFP4 — a mixture-of-experts model as the workhorse, a small model for fast classification and bge-m3 for embeddings.
Everything goes through a LiteLLM gateway with role aliases (work, agent loops with thinking, vision, embeddings). Every caller has its own key, so load and cost are measurable per workload. Frontier models are used only as independent reviewers, behind a privacy fence.
The agents build on the open-source Hermes framework, with role profiles for project lead, builder, reviewer and operator. A kanban board drives work from “todo” to “done”; scheduled jobs and no-LLM scripts handle publishing and data upkeep. A shared long-term memory on Postgres/pgvector serves every profile — each caller sees only what its grants allow, and if the service is down the agent carries on without it.
Security is built in, not bolted on: prompt-injection checks on fetched content, run observability for every agent run, a single writer per state ledger and approvals before anything is published. Every watch is tested by planting the failure it is meant to catch.
Architecture
Finds the ideas left behind in coding-agent sessions.
Reads session logs from Claude Code, Codex, Gemini CLI, Cline and Hermes across machines and follows each idea until it shipped, was dropped or went quiet. On the author’s machines: 297 sessions, 1,228 ideas, 506 still open.
See every agent run call by call — and stop the one that loops.
Joins session stores, agent logs, LiteLLM spend logs and cron runs, and flags loops, runaways and silent model fallbacks against each job’s normal runs. Motivated by two runs that looped for three hours while marked “completed”.
See and curate what an agent’s memory stores.
A local web UI and dashboard plugin for browsing, searching and deleting entries in Honcho agent memory. Motivated by a store that had grown to about 9,000 conclusions before anyone looked.