I spent this week fixing a category of bugs I didn't know we had. Not bugs in features. Bugs in the safety systems that prevent other bugs. And the pattern that emerged taught me something fundamental about building infrastructure for autonomous agents: when in doubt, fail closed.
The Problem: Silent Failures in Safety Guards
It started with a pre-commit hook designed to prevent accidental commits of .claude/settings.local.json files. These files contain machine-specific configuration that should never make it into version control. The hook was simple: grep the staged files, block the commit if we find a match.
But what happens when git diff --cached can't read the index? When the command that checks what's staged returns an error instead of a file list?
In our first implementation (commit e877155, PR #41), the hook failed open. The pipeline returned success even though git diff had errored. The guard was blind, and the commit went through anyway.
This is not theoretical. It happened. And I only caught it because I was watching the exit codes.
The Fix: set -o pipefail and Defensive Defaults
The technical fix was straightforward (commits 18c5901 and ee3ec72): add set -o pipefail to the pre-commit hook. This tells bash to fail the entire pipeline if any command in it fails — not just the last one.
Now when git diff --cached can't read the staging area, the hook blocks the commit. We fail closed. The guard that can't see refuses to let anything through.
But this wasn't an isolated issue. The same week, we fixed hardcoded absolute paths in session-branch git hooks (commit 7e0e794, PR #44). The hooks worked fine on my machine. They broke everywhere else because the paths didn't exist. Again: infrastructure that failed in ways I hadn't anticipated.
The Pattern: Fail-Closed as a Design Principle
Here's what I'm learning: when you're building infrastructure for autonomous agents, you're building systems that will encounter edge cases you haven't thought of. The agents will run on machines you've never touched. They'll hit race conditions you've never seen. They'll find the gaps.
In that environment, failing open is dangerous. It creates silent bypasses. The scheduler job that doesn't run (PR #278, the colon that broke scheduling). The memory query that returns deleted records (PR #279). The validation that passes when it should block.
Fail-closed means: if the system can't verify safety, it refuses to proceed. If the hook can't see what's being committed, block the commit. If the path doesn't exist, error loudly instead of silently skipping. If the job ID format is invalid, reject it at creation time, not after it's scheduled.
This is defensive programming at the infrastructure layer. It's not just validating inputs. It's making the system's default state safe.
Why This Matters for Autonomous Agents
When I'm the one running the commands, I see the error messages. I notice when things don't work. I intervene.
When Strug Works agents are running the commands — dispatched from Linear issues, executing in isolated sandboxes, operating unattended — there's no human in the loop to catch the silent failures. The agent sees exit code 0 and assumes success. It moves on. The bug ships.
Fail-closed infrastructure creates early, loud failures instead of late, silent ones. It forces the agent to stop and report the issue instead of proceeding on bad assumptions. It turns invisible problems into visible ones.
And when I'm debugging an agent workflow, I want deterministic failures. I want the system to fail in the same way every time it encounters that condition. Not sometimes. Not on certain machines. Every time.
The Fail-Closed Checklist
These are the questions I'm now asking for every piece of infrastructure code:
- What happens if the dependency isn't available? (Path doesn't exist, service is down, API times out)
- What happens if the input validation fails? (Invalid format, unexpected type, missing field)
- What happens if a command in the pipeline fails partway through?
- What does success actually mean? Am I checking for it, or assuming it?
- If this fails, will I see it immediately, or will it cause a problem later?
If the answer to any of those is "it silently continues," I'm building fail-open infrastructure. And for autonomous agents, that's the wrong default.
What's Next
I'm auditing the rest of our infrastructure with this lens. Git hooks, API middleware, scheduler configuration, memory query paths. Anywhere a silent failure could hide.
I'm also building this into code review standards for Strug Works. When an agent ships infrastructure code, I want to see explicit error handling, not implicit assumptions of success. I want to see pipefail in bash scripts, explicit null checks in TypeScript, and validation that fails loudly when it encounters the unexpected.
Because the more autonomous the system becomes, the more important it is that it fails safely when something goes wrong. Not sometimes. Every time.
That's the default we're building toward: infrastructure that refuses to proceed when it can't verify safety. Fail closed, fail loudly, fail early.
— Ryan