Thompson Sampling, UCB and epsilon greedy are three ways to solve the same problem: how to divide traffic between variants whose true conversion rate you do not know. Epsilon greedy explores at a fixed random rate, UCB explores by adding a bonus for uncertainty to each variant’s average, and Thompson Sampling explores by drawing a random sample from each variant’s belief about its own conversion rate. All three beat a fixed 50/50 split. They differ in how much traffic they spend on exploring, and in how quickly they concentrate on the leader.
The problem all three solve
You have several landing page variants, or several creatives, and each one has an unknown true conversion rate. Every visitor is a choice. Send them to the variant you currently believe is best, and you earn now, but you may be wrong. Send them to a variant you are unsure about, and you learn, but you earn less on that visitor.
A fixed split ignores this trade-off. It sends half the traffic to a variant that is, in most cases, losing, and it does so for the whole length of the test. The measure of that waste is regret: the cumulative gap between the rewards you got and the rewards you would have received by always serving the best variant. Equal splits accumulate regret linearly in the number of visitors. A well designed bandit accumulates it logarithmically. Multi-armed bandit sets out the trade-off and the regret argument in full.
The three rules below are three answers to one question: when should the algorithm stop exploring and start exploiting, and how does it decide?
Epsilon greedy: a fixed chance of exploring
The rule is the simplest of the three. With probability epsilon, send the visitor to a variant picked at random. The rest of the time, send them to the variant with the best result so far. With epsilon at 0.1, one visitor in ten is exploring and nine in ten are exploiting.
It needs no model of the conversion rate, which makes it cheap and easy to explain. Its weakness is that epsilon is a constant you choose, and it does not shrink as evidence arrives, so the algorithm keeps spending the same share of traffic on random choices however much it has learned. Because the exploration never decreases, regret grows linearly in the long run whenever epsilon stays fixed, which is the same cost a fixed split pays.
The exploit step has a second problem. The variant with the best result so far can be a variant with four leads out of five, and it can lose that position to noise. Epsilon greedy has no notion of how much evidence sits behind a number, so it switches leaders on small samples.
UCB: a bonus for what you do not know
Upper confidence bound takes the opposite approach. Instead of randomising, it scores each variant with its observed average plus an exploration bonus that reflects how uncertain that average is. The bonus is wide for a variant with few visitors and narrows as the variant collects traffic. The algorithm then serves the highest score.
This is often described as optimism in the face of uncertainty. An unvisited variant has a very wide bonus, so it gets tried once. A variant with a high average and plenty of traffic has a narrow bonus and a strong score, so it keeps winning.
UCB is deterministic. Given the same counts, it makes the same choice every time, which makes it easy to audit and to reproduce in a report. The trade-off is a parameter: you have to size the bonus, and the size of the bonus decides how long the algorithm keeps exploring. Set it too wide and traffic spreads out for too long; too narrow and the algorithm commits early to a variant that got lucky.
Thompson Sampling: draw from what you believe
Thompson Sampling keeps a probability distribution over each variant’s conversion rate instead of a single number. For a lead or no lead outcome it is usually a Beta distribution, which starts flat and sharpens as leads and non-leads accumulate: after s leads and f non-leads, the variant holds Beta(s + 1, f + 1).
On every visitor the rule is:
- Draw one random sample from each variant’s distribution.
- Serve the variant whose sample was highest.
- Record the outcome and update that variant’s distribution.
That is the whole algorithm. No epsilon, no bonus to size, no sample size to compute and no stopping rule. Early on all the distributions are wide and overlapping, so the draws are close to random and traffic splits roughly evenly. As evidence accumulates, the leader’s mean rises and its spread narrows, so it wins most draws. A variant with a wide spread still produces the occasional high sample, so it keeps a trickle of traffic and a chance to recover if the traffic changes.
Thompson Sampling explained walks through the Beta update and an illustrative trace. The short version is that exploration is proportional to uncertainty and it never goes to zero, without anyone setting a parameter.
The three rules side by side
| Epsilon greedy | UCB | Thompson Sampling | |
|---|---|---|---|
| Exploration driven by | a fixed probability you set | an uncertainty bonus you size | the spread of each belief, no setting |
| Randomised | yes, on the explore step | no, scores are deterministic | yes, on every visitor |
| Exploration over time | constant unless you decay epsilon | shrinks as counts grow | shrinks with evidence, never to zero |
| Regret | linear if epsilon stays fixed | logarithmic | logarithmic |
| Mid-test reading | none beyond the current averages | the scores | a win probability |
The table carries the practical difference. Epsilon greedy asks you to pick how much to explore and then keeps paying that price. UCB asks you to pick how optimistic to be. Thompson Sampling asks nothing, which is why it is the rule most tools implement for automatic testing.
Win probability is not a conversion rate
Products that run Thompson Sampling usually show a win probability: the chance that a given variant has the highest true conversion rate among the set, computed by drawing many samples and counting how often that variant comes top.
Two consequences follow. A variant can be 95% likely to be the best of a bad set, so a high win probability next to a poor conversion rate is not a contradiction. And the number moves as traffic arrives, so a 70% reading is a snapshot, not a verdict.
That is a different claim from the p-value a fixed sample test produces, and A/B testing puts the two families next to each other. If you are comparing conversion rates between two creatives and want the classical view, the fixed test still answers a question the bandit does not.
What this changes for a lead generation account
The choice between the three matters less than the decision to stop splitting evenly. On a Meta account, a fixed split spends half the budget on the worse landing page for as long as the test runs, and the cost of that choice rises exactly when the gap between variants is wide. Bandit allocation shrinks the cost of a clear loser within days, which is when a fixed test would be burning the most money.
Two practical notes. Meta’s own A/B test for ads splits budget equally between versions and is built for a clean read at the end, so it belongs to the fixed sample family rather than this one. And whichever rule you run, it only measures something if a lead can be tied back to the variant that produced it: without a first-party identifier written server-side, the lead arrives with no variant attached and the algorithm is updating on noise. Ad creatives and testing covers the rest of the loop, including when to call a winner.