- Headline result
- 14 self-improving automations
- Client
- My own operations
- My role
- Designed and built it
- Scope
- 14 self-improving automations, two-lane approval, every pipeline change backtested
- When
- July - September 2026
The problem
A normal automation is a straight line. Run, exit, hope. When it drifts, nobody notices until the output is wrong. The only way it ever improves is a human remembering to look at it.
The obvious fix, letting an AI agent change things on its own, is how you get a broken production system at 3am. The model is confident, the change looks fine and nobody checked.
I wanted every system I run to behave like a good team member: check its own work, fix what it can, report what it cannot and get a little better every week, without the risk of an unchecked edit on production.
What I did
A loop pattern with a four-step maturity ladder and a fleet of systems built on it. L0 self-run → L1 self-heal → L2 self-report → L3 self-improve.
L1: every cron is wrapped. A checkup decides done, not exit code 0. Transient failures retry with backoff. Real failures alert. L2: every run writes a heartbeat and a structured outcome that a live systems board reads. L3: a weekly audit on the strongest model reads each loop's own reports and proposes its next improvement.
On top of the ladder sits a two-lane approval board. Backend and knowledge changes go through a deterministic guard chain and ship themselves with a version bump. Creative or voice changes always wait for my sign-off. Either way, a failed check reverts the change before it reaches production. Each loop keeps a version history, so I can see what it changed about itself and when.
How it works
Two lanes: code ships itself, voice waits for me
A whitelist of what the auto lane may touch, a syntax check, a secret scan and an auto-revert on any failure. That lane ships on its own. Anything that changes how I sound or what I say to a client waits on the approval board, one click to accept.
Propose, then prove with a backtest
For the lead pipeline, a proposed rubric change is replayed against the full history first. It only ships if it provably would have scored past leads better. I see the delta before I approve. A change that would have made things worse never reaches the board.
Judgment on the strongest model, bulk on a cheaper one
Audits and approval decisions run on the best model available. Building and routine content run on a cheaper one. A monitor keeps the model registry current when a newer model ships, with a fallback ladder so no loop dies when one model is capped.
Micro loops per step, one macro loop over all of them
Each pipeline step has its own loop: scraping, scoring, research, content. A weekly macro loop reads all of them, spots cross-step issues and models where the human is the bottleneck.
It audits the way I work too
The same loop reads a weekly analysis of my own sessions and builds small non-blocking fixes for recurring friction. The system that improves my automations also improves my instructions to it.
Results
- 14 self-improving loops live, each with a version history
- Low-risk changes ship automatically through the guard chain, anything material waits for one click
- Any change that fails a check auto-reverts before it reaches production
- Pipeline changes backtested against the full history before they ship, the delta shown to me first
- The loop system audited its own audit and found a macro loop that had been dead for 16 nights while reporting healthy, the exact class of failure it now catches
What this means for your business
People ask how I let AI touch systems unattended. It is not trust in the model. It is whitelists, secret scans, backtests and auto-revert, with a human on anything that needs taste. That split is the whole design. It is the same split I would set up for a team that wants AI doing real work without anyone babysitting it. Happy to grab a call to see which of your processes could run as a loop.
Tools
- Python
- Claude Code
- Supabase
- Next.js
- Cloud server
- Telegram
Skills
- AI Automation
- AI Agents
- Autonomous Systems
- Human-in-the-Loop Design
- Business Process Automation
- Continuous Improvement
- AI Integration
- Operations Manager
Questions
Is it safe to let AI agents change production systems?
Only inside a cage. Mine can touch a whitelisted set of files, every change passes a syntax check and a secret scan, pipeline changes are backtested against history and anything that fails reverts on its own. Changes to voice or customer-facing content never ship without a human click.
What is a self-improving automation loop?
An automation that does four things: runs on a schedule, heals its own transient failures, reports a structured outcome somewhere a human can see and periodically reads its own results to propose its next improvement. Most automations stop at step one.
How do you stop an AI agent from making a bad change?
Do not rely on the model to judge itself. Deterministic checks decide: whitelist, syntax, secrets, a backtest where there is history to replay. If any check fails the change reverts before production. The model proposes, the code decides, the human approves the rest.
Which parts should always need human approval?
Anything a customer sees, anything that spends money and anything that changes how you sound. Backend plumbing, data fixes and tuning can ship on their own behind the guard chain. Keeping those two lanes separate is what makes the auto lane safe.