Background
The homelab is small by enterprise standards but big enough that the relationships between services had quietly become tribal knowledge. Which container depends on which database? If Home Assistant goes down, what else breaks? The infra-stack app already rendered a layered view of containers and endpoints, but nothing in it described how services actually depended on each other.
The fix was to stop encoding that knowledge in app code and start treating infrastructure as data: a single catalog.yaml as the source of truth for services, hosts, endpoints, and hard runtime dependencies — consumed by a new relationships view, an audit script, and a generated docs page.
What I built
The catalog — One YAML file describing every service in the homelab: which host it runs on, what HTTP endpoints front it, what other services it depends on at runtime. A hand-rolled validator checks the schema, rejects duplicate IDs, and catches dependency cycles using a depth-first traversal with three-color marking. About eighty lines of Python and eleven unit tests, no external schema dependency.
The relationships view — A new /graph page that lays the homelab out as a left-to-right column-per-layer DAG. Edges are server-computed; the browser receives node positions and curve geometry ready to render as SVG. Click any node and its ancestor and descendant chains highlight, with the selection surviving the thirty-second background refresh. Hover shows the full dependency tree for that service.
The reconciler — A script that diffs the catalog against three live sources: Gatus endpoint configs, Docker container names on the host, and Caddy reverse_proxy targets. It surfaces drift in either direction — services in the catalog that no longer exist in the wild, and services running in the wild that the catalog does not know about. Achieving “clean” across all three sources became the bar for catalog completeness.
The docs generator — Writes a Markdown table of every service and host into the homelab workspace’s SERVERS.md. The infra-stack repo is separate from the homelab workspace repo, so this is the first piece of tooling to do a cross-repo write — the generator is the one place that knows the path to the workspace.
How it paid off
The view shipped on a Saturday morning. Within hours of going live, the first thing it surfaced was a real outage that had been silent for forty-six hours.
The graph reported affected: ['homebridge']. Following the dependency chain showed homebridge healthy on its own but pointing at home-automation, which had one failing Gatus endpoint: the Zigbee2MQTT bridge connection state. The Home Assistant REST API confirmed the bridge had gone silent on a previous afternoon during a Home Assistant Core update — fifty seconds into a fifty-nine-second restart window, the Z2M add-on stopped responding and never came back. Every Zigbee entity in the house had been unavailable since.
Without the catalog’s depends_on edges, “homebridge is healthy” and “home-automation has one failing check” would have read as two unrelated facts in the dashboard. The view connected them. Diagnosis to fix took minutes.
The deeper finding came from grepping the supervisor logs: there was no explicit stop or crash entry for Z2M anywhere in the forty-six-hour window. The container was running the whole time — the Node.js process inside it was wedged. Container-level watchdogs cannot see this failure mode.
Three layers of defense
Catching the symptom was not enough. Three layers went in to prevent a repeat:
- Start on boot in the Home Assistant add-on UI — covers full Home Assistant reboots.
- Supervisor watchdog — restarts the add-on if its container exits.
- A Home Assistant automation watching the MQTT bridge connection state, set to trigger
hassio.addon_restartif the entity staysofffor five minutes, with a push notification.
Only the third layer covers the “container alive, process dead” case that caused this particular outage. For community add-ons without a Docker HEALTHCHECK directive, that third layer is the actual fix.
How it was built
The whole effort — spec, plan, schema, validator, graph view, reconciler, docs generator — moved from idea to merged pull request in one working session of roughly sixteen commits.
The workflow used a controller-and-subagent pattern: a controller agent drove the plan, dispatched fifteen implementer subagents for the discrete tasks, and ran more than thirty review subagents in two passes (spec compliance, then code quality) between every task. The reviewers caught four real issues the implementers missed: unused imports, vacuous tests that excluded the very hosts they meant to cover, pseudo-services silently included in graph output, and an XSS footgun lurking in a code path planned for a later phase.
Forty-two tests, a validator-clean catalog across three live sources, and a server-computed graph layout with zero client-side traversal. The view shipped the next morning.
Why it matters
The graph view is the visible piece, but the load-bearing decision was treating the catalog as the source of truth instead of growing the encoded knowledge inside the app. Everything else — the reconciler, the docs, the cascade view — became a consumer of that file. Future work that needs to know about service topology has one place to read from and one place to update.
It also closed a working assumption about Home Assistant monitoring: the supervisor watchdog catches container failures, but the most insidious add-on failure mode is a wedged process inside a healthy-looking container. That is a different problem and needs a different layer of defense. The graph view was the thing that surfaced the gap.