Once the Research Engine said an idea was worth testing, I needed to test it fast. The Experiment Engine gets a real website up and in front of real users as quickly as possible.
The problem
Validating demand for real means putting something in front of real people and watching what they do — a live landing page, a real ad campaign, actual behaviour. Done by hand, that's days of work per test: writing copy for each audience, designing and building the pages, setting up a campaign, wiring up analytics, then collecting and making sense of the results. It's so much effort that you run far fewer experiments than you should, and cut corners on the ones you do run. The bottleneck was never ideas to test — it's the cost of testing them properly.
My approach
Standardise the experiment so the machine can run it end to end. I give it the variables — the audiences, the value propositions, the angles I want to test — and it applies the same rigorous testing methodology every time, so results are comparable across experiments instead of a pile of one-offs. I talk through what I actually want to learn; it handles everything from there. A consistent method, run consistently, is worth far more than a clever one-off.
What I built
An agent pipeline that takes an experiment from idea to results with me out of the middle. I hand it the test variables and we agree on the experiment. Before it spends anything, it checks a ledger to see whether this test has already been run — no paying twice for the same answer. Then the agents use everything gathered, plus a component library and a set of themes, to build one or more landing pages and a matching Google Ads campaign. It pushes the landing page live to production and — where an ad account is connected — uploads and runs the campaign. When the campaign finishes, it pulls the behavioural data from PostHog and the ad data from Google, analyses it, and writes the whole thing up in Confluence for me to weigh in on. Idea in, live test out, a clean readout back. What used to take days or weeks — building the pages, wiring the campaign, gathering and reading the data — now takes minutes.
Why it mattered
It collapses the cost of a real experiment from days to almost nothing, which changes the behaviour that matters: I can run many more tests, properly, than I ever would by hand. And because every experiment uses the same method and lands in the same place, the results accumulate into something I can compare and learn from, instead of a heap of unrepeatable one-offs. It's the step that turns "I think this might work" into evidence.
What I learned
The unlock was standardising the method, not automating a task. Once every experiment runs the same way — same structure, same tracking, same write-up — the machine can own the whole thing, and the results become comparable across time. The ledger taught me a smaller but real lesson too: half of doing this at scale is refusing to redo work you've already done.