Ad creatives and testing

How is Thompson Sampling different from UCB and epsilon greedy?

Thompson Sampling, UCB and epsilon greedy solve the same explore and exploit trade-off in three different ways. How the allocation rules differ.

Published

Thompson Sampling, UCB and epsilon greedy are three ways to solve the same problem: how to divide traffic between variants whose true conversion rate you do not know. Epsilon greedy explores at a fixed random rate, UCB explores by adding a bonus for uncertainty to each variant’s average, and Thompson Sampling explores by drawing a random sample from each variant’s belief about its own conversion rate. All three beat a fixed 50/50 split. They differ in how much traffic they spend on exploring, and in how quickly they concentrate on the leader.

The problem all three solve

You have several landing page variants, or several creatives, and each one has an unknown true conversion rate. Every visitor is a choice. Send them to the variant you currently believe is best, and you earn now, but you may be wrong. Send them to a variant you are unsure about, and you learn, but you earn less on that visitor.

A fixed split ignores this trade-off. It sends half the traffic to a variant that is, in most cases, losing, and it does so for the whole length of the test. The measure of that waste is regret: the cumulative gap between the rewards you got and the rewards you would have received by always serving the best variant. Equal splits accumulate regret linearly in the number of visitors. A well designed bandit accumulates it logarithmically. Multi-armed bandit sets out the trade-off and the regret argument in full.

The three rules below are three answers to one question: when should the algorithm stop exploring and start exploiting, and how does it decide?

Epsilon greedy: a fixed chance of exploring

The rule is the simplest of the three. With probability epsilon, send the visitor to a variant picked at random. The rest of the time, send them to the variant with the best result so far. With epsilon at 0.1, one visitor in ten is exploring and nine in ten are exploiting.

It needs no model of the conversion rate, which makes it cheap and easy to explain. Its weakness is that epsilon is a constant you choose, and it does not shrink as evidence arrives, so the algorithm keeps spending the same share of traffic on random choices however much it has learned. Because the exploration never decreases, regret grows linearly in the long run whenever epsilon stays fixed, which is the same cost a fixed split pays.

The exploit step has a second problem. The variant with the best result so far can be a variant with four leads out of five, and it can lose that position to noise. Epsilon greedy has no notion of how much evidence sits behind a number, so it switches leaders on small samples.

UCB: a bonus for what you do not know

Upper confidence bound takes the opposite approach. Instead of randomising, it scores each variant with its observed average plus an exploration bonus that reflects how uncertain that average is. The bonus is wide for a variant with few visitors and narrows as the variant collects traffic. The algorithm then serves the highest score.

This is often described as optimism in the face of uncertainty. An unvisited variant has a very wide bonus, so it gets tried once. A variant with a high average and plenty of traffic has a narrow bonus and a strong score, so it keeps winning.

UCB is deterministic. Given the same counts, it makes the same choice every time, which makes it easy to audit and to reproduce in a report. The trade-off is a parameter: you have to size the bonus, and the size of the bonus decides how long the algorithm keeps exploring. Set it too wide and traffic spreads out for too long; too narrow and the algorithm commits early to a variant that got lucky.

Thompson Sampling: draw from what you believe

Thompson Sampling keeps a probability distribution over each variant’s conversion rate instead of a single number. For a lead or no lead outcome it is usually a Beta distribution, which starts flat and sharpens as leads and non-leads accumulate: after s leads and f non-leads, the variant holds Beta(s + 1, f + 1).

On every visitor the rule is:

  1. Draw one random sample from each variant’s distribution.
  2. Serve the variant whose sample was highest.
  3. Record the outcome and update that variant’s distribution.

That is the whole algorithm. No epsilon, no bonus to size, no sample size to compute and no stopping rule. Early on all the distributions are wide and overlapping, so the draws are close to random and traffic splits roughly evenly. As evidence accumulates, the leader’s mean rises and its spread narrows, so it wins most draws. A variant with a wide spread still produces the occasional high sample, so it keeps a trickle of traffic and a chance to recover if the traffic changes.

Thompson Sampling explained walks through the Beta update and an illustrative trace. The short version is that exploration is proportional to uncertainty and it never goes to zero, without anyone setting a parameter.

The three rules side by side

Epsilon greedyUCBThompson Sampling
Exploration driven bya fixed probability you setan uncertainty bonus you sizethe spread of each belief, no setting
Randomisedyes, on the explore stepno, scores are deterministicyes, on every visitor
Exploration over timeconstant unless you decay epsilonshrinks as counts growshrinks with evidence, never to zero
Regretlinear if epsilon stays fixedlogarithmiclogarithmic
Mid-test readingnone beyond the current averagesthe scoresa win probability

The table carries the practical difference. Epsilon greedy asks you to pick how much to explore and then keeps paying that price. UCB asks you to pick how optimistic to be. Thompson Sampling asks nothing, which is why it is the rule most tools implement for automatic testing.

Win probability is not a conversion rate

Products that run Thompson Sampling usually show a win probability: the chance that a given variant has the highest true conversion rate among the set, computed by drawing many samples and counting how often that variant comes top.

Two consequences follow. A variant can be 95% likely to be the best of a bad set, so a high win probability next to a poor conversion rate is not a contradiction. And the number moves as traffic arrives, so a 70% reading is a snapshot, not a verdict.

That is a different claim from the p-value a fixed sample test produces, and A/B testing puts the two families next to each other. If you are comparing conversion rates between two creatives and want the classical view, the fixed test still answers a question the bandit does not.

What this changes for a lead generation account

The choice between the three matters less than the decision to stop splitting evenly. On a Meta account, a fixed split spends half the budget on the worse landing page for as long as the test runs, and the cost of that choice rises exactly when the gap between variants is wide. Bandit allocation shrinks the cost of a clear loser within days, which is when a fixed test would be burning the most money.

Two practical notes. Meta’s own A/B test for ads splits budget equally between versions and is built for a clean read at the end, so it belongs to the fixed sample family rather than this one. And whichever rule you run, it only measures something if a lead can be tied back to the variant that produced it: without a first-party identifier written server-side, the lead arrives with no variant attached and the algorithm is updating on noise. Ad creatives and testing covers the rest of the loop, including when to call a winner.

Questions on this topic

All questions on this topic →

FAQ

Is Thompson Sampling always better than UCB?+

No. Both keep regret logarithmic, so neither wins on the theoretical bound. UCB is deterministic and easier to audit, because the same counts always produce the same choice. Thompson Sampling is easier to extend when you have a prior, and it handles many variants without a tuning parameter.

Does epsilon greedy still have a use?+

It is a fine baseline and the easiest rule to explain to a client. It becomes expensive when epsilon stays fixed, because the algorithm keeps spending a constant share of traffic on random choices no matter how much evidence it has.

Do I need a statistics background to run a bandit?+

No, if you use a tool that runs it for you. You do need to understand two ideas: the traffic split adapts on its own, and the win probability you read mid-test is a statement about your evidence, not a forecast of the conversion rate.

How many variants can I test at once?+

Each extra variant is another arm that has to be explored, so start with two or three. With a small lead volume, a fourth variant mostly adds noise and delays the point where the leader gets most of the traffic.

Does this apply to ad creatives or only to landing pages?+

It applies to any choice where every visitor can only be sent to one option, so both. Meta runs its own ad tests with equal budgets per version, which is the fixed split these rules replace.

Ready to find out what
converts, for real?

Free to start.

Start free →