A classic A/B test has an uncomfortable property: it spends half your money on the page you already suspect is worse.
You choose a sample size in advance, split traffic 50/50, and wait. If the test needs 4,000 visitors per variant and your losing page converts at half the rate, you have knowingly paid for thousands of visitors on the wrong page — and you cannot stop early without invalidating the statistics.
What DropCPL does instead
DropCPL treats the choice between landing pages as a multi-armed bandit problem, and solves it with Thompson Sampling.
Each landing page variant holds a probability distribution over its conversion rate. On every visitor:
- Draw a sample from each variant’s distribution.
- Serve the variant with the highest drawn value.
- Update that variant’s distribution with the outcome (lead or no lead).
Early on, the distributions overlap heavily, so traffic is split roughly evenly — the exploration phase. As evidence accumulates, the winner’s distribution separates, and the draw increasingly favours it. Traffic shifts on its own.
The test does not stop when it reaches significance. It never stops reallocating.
Why this matters for lead generation
The cost of a classic test is not the statistics, it is the traffic you spend on the loser. With a bandit, that cost shrinks continuously, and it shrinks most exactly when the gap between the variants is large — which is when the savings are worth the most.
The practical consequence: you can keep more pages in rotation, because running a test is no longer an expensive, planned event. Add a variant, and the allocation handles it.
What you read in the product
- Conversion rate per variant, with the interval, not just a point estimate.
- Win probability for each variant — how confident the model is that this is the best of the set.
- Allocation share — where the traffic is actually going right now, which is the direct answer to “how much am I still paying to explore?”.
The honest limits
Thompson Sampling does not fix a broken test. If a variant gets almost no traffic, its evidence accumulates slowly — so a page that starts badly is not abandoned forever, but it does recover slowly. And if your lead volume is very low (a handful of leads per week), no allocation method will produce a confident answer in a useful amount of time. The honest response there is to test bigger differences, not to test more.
Related
- Landing pages in HTML — what gets tested
- Creative attribution — the matrix the winner feeds into
- Thompson Sampling explained — the algorithm, with the maths