Landing page and offer split testing
A split test checks a specific hypothesis about an offer or landing page. Define the control, metric and stopping conditions before looking at results.
Hypothesis and control group
Change one factor, such as the landing page while keeping the offer and source the same. The first variant is the control. Compare concurrent audiences with matching geography and device restrictions. Changing the source, offer and page together makes the cause of any difference unclear.
Fixed traffic shares for A/B testing
An experiment needs planned shares. A bandit adapts allocation to results and serves a different purpose. When an inotrack test starts on a bandit or priority flow, it temporarily uses a fixed split and restores the previous mode when the test ends. Check actual traffic by variant because availability and caps can affect the sample.
CR, EPC and enough data
CR relates conversions to clicks; EPC relates revenue to clicks. The test verdict uses approval proportions, while EPC is displayed for reference. The default minimum is 200 clicks per variant, a technical threshold rather than a universal sample size. Allow time for approvals and plan around the effect you need to detect.
Multiple landing pages or offers
Compare two to six offers or landing pages against the first variant. Holm–Bonferroni correction handles multiple comparisons. Avoid adding variants simply because an early result looks promising: unplanned repeated checks can make random variation look conclusive.
Winner, no difference or insufficient data
The report distinguishes a winner, no difference, insufficient data and multiple improvements without a single clear winner. No difference does not prove equivalence. Automatic stopping is evaluated when the test report is opened; account for that when monitoring an experiment.
Apply the split test result
After confirmation, applying the winner assigns it the combined weight of the tested variants and leaves the others at zero. The flow stays on a fixed split. Check the route and postback afterwards. Findings apply to the audience and period tested and do not guarantee future ROI.
Questions
How many clicks does an A/B test need?
There is no universal count. Plan for the baseline conversion rate, effect size and conversion delay; the default 200-click threshold alone is insufficient.
Is a bandit an A/B test?
A bandit adapts allocation, while a fixed split supports a controlled comparison. Choose the approach to match the question.
Read next
Run inotrack on your own server
One command on a clean VPS, 10–15 minutes. Get your key right after you pay on the site, or a free one for 7 days.
Questions? Message us on Telegram: @SmokeJung.