MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas
MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas 2015 Testing Year in Review: 12 months of macro- and micro-yes experiments Posted: 29 Feb 2016 12:01 AM PST Tweet Earlier this month, I brought you the first in a series of 2015 Testing Year in Review blog posts. For this next post, we'll be looking at tests using both KPI macro-yes(s) (conversions) and micro-yes(s) (clickthrough). Here are results of the 2015 tests we ran based on the primary test metric and level of confidence achieved. Anything under 95% level of confidence (LOC) is considered inconclusive, meaning the test did not have statistically significant findings. We want all the tests to be above 95% LOC because then we can say (with 95% confidence) that the changes we made are having the impact we want. The size of the boxes represents the range of LOCs, and the black lines are the midpoint, where half the data is above, and half below t...