Agent & Automation Tooling

The problem

Two separate frustrations pointed the same direction.

The first was that questions about the infrastructure were being answered from memory. “What depends on this service?” had a correct answer written down somewhere, but retrieving it meant reading a very long document, so in practice people guessed — including me.

The second was that recovery was manual. A container failing its health check at 3am is not an interesting problem; it is a restart. But it was a restart someone had to be awake for.

The approach

Two Model Context Protocol servers, both read-only by design. The first parses a large infrastructure document into a queryable interface — hosts, services, network details, free-text search — so questions are answered from the document of record instead of recollection. The second talks to a storage vendor’s undocumented web API.

Around twenty-one automation workflows handle what would otherwise be pages. Failed health checks trigger restarts behind a circuit breaker, so a genuinely broken service does not loop forever. DNS fails over between redundant servers. Stalled transfers get detected and cleared. Scheduled digests summarise overnight activity.

A weekly security pipeline runs five scanners across the production application — dependency vulnerabilities, static analysis, secret detection and container image scanning — diffed against a tracked baseline so only new findings surface.

What was actually hard

The vendor API wrapped its login in RSA encryption and documented none of it. Building the second server meant reverse-engineering the authentication handshake from browser traffic, then covering the result with 25 unit tests and a protocol-level regression test.

That last test exists because of a genuinely instructive bug: the server connected successfully and exposed zero tools. The cause was two framework instances being created where one was intended. A component that reports healthy while doing nothing is the exact failure mode worth writing a test for, because nothing else will catch it — every signal says fine.

That principle drove the rest. Automation that silently does nothing is worse than no automation, so every workflow reports and the absence of a report is itself an alert. The security pipeline follows the same rule: a scanner that fails to run produces a loud failure rather than a misleading clean result. A check that fails open teaches you nothing except false confidence.

The outcome

The repository carries its own operating manual — 34 task-specific skills, seven custom agent definitions, and 37 rule files covering deployment sequencing and the failure classes that must not repeat.

Treating that configuration as versioned infrastructure, reviewed and audited like the systems it manages, is what makes automated changes trustworthy enough to leave running unattended.