Giving AI agents an environment they can test
Agents can only close the loop when they can run, seed and observe the system on their own. I built the test data, production-like data and payment simulation that made that possible.
- Context
- B2B marketplace platform
- My role
- Hands-on builder
- When
- 2026
- end-to-end business scenarios seeded on demand
- 100+
- personal-data columns masked in prod-like data
- 30+
- DevOps skills packaged as a Claude Code plugin
- 8
The insight
Most failed AI agent projects don’t fail because of the model. They fail because the agent can’t see or run the system it’s changing. I treat making a platform agent-ready as its own workstream.
What I built
- Scenario seeding. An API-driven tool that creates fresh accounts for every role and walks through realistic business flows. It goes after the biggest QA cost: preparing test data.
- Production-like data, safely. A weekly clone of the production read replica, with personal data (emails, bank accounts) masked, exposed as a seed API so every feature sandbox can start with realistic data.
- Payment-provider simulator. A stateful mock of the payment provider’s REST API with realistic webhooks, plus a control API for steps that normally need a human or a bank (strong customer authentication, KYC, incoming wire transfers).
- Platform operations as AI skills. A Claude Code plugin with 8 skills (deploy, status, logs, sandbox lifecycle, read-only Kubernetes inspection), so any engineer, or any agent, can operate environments safely. Secrets come from 1Password and never appear in prompts.
Why it matters
These pieces turned “AI writes the code, a human checks everything” into “AI writes, tests and verifies on a realistic environment, and a human approves”. The second is what actually scales.