WiderFunnel Conversion Optimization Blog

Posted: 08 Mar 2016 05:25 AM PST

The potential for your marketing improvement depends on the strength of your testing hypotheses.

But where are you getting your test ideas from? Have you been scouring competitor sites, or perhaps pulling from previous designs on your site? The web is full of ideas and you're full of ideas – there is no shortage of inspiration, that's for sure.

Coming up with something you want to test isn't hard to do.

Coming up with something you should test can be hard to do.

Hard – yes. Impossible? No. Which is good news, because if you can't create hypotheses for things that should be tested, your test results won't mean mean much, and you probably shouldn't be spending your time testing.

Taking the time to write your hypotheses correctly will help you structure your ideas, get better results, and avoid wasting traffic on poor test designs.

With this post, we're getting advanced with marketing hypotheses, showing you how to write and structure your hypotheses to gain both business results and marketing insights!

By the time you finish reading, you'll be able to:

Distinguish a solid hypothesis from a time-waster, and
Structure your solid hypothesis to get results and insights

To make this whole experience a bit more tangible, let's track a sample idea from…well…idea to hypothesis.

Let's say you identified a call-to-action (CTA)* while browsing the web, and you were inspired to test something similar on your own lead generation landing page. You think it might work for your users! Your idea is:

“My page needs a new CTA.”

*A call-to-action is the point where you, as a marketer, ask your prospect to do something on your page. It often includes a button or link to an action like “Buy”, “Sign up”, or “Request a quote”.

The basics: The correct hypothesis format

A well-structured hypothesis provides insights whether it is proved, disproved, or results are inconclusive. You should never phrase a hypothesis as a question. It should be written as a statement that can be rejected or confirmed.

Further, it should be a statement geared toward revealing insights – with this in mind, it helps to imagine each statement followed by a reason:

Changing _______ into ______ will increase [conversion goal], because:
Changing _______ into ______ will decrease [conversion goal], because:
Changing _______ into ______ will not affect [conversion goal], because:

Each of the above sentences ends with 'because' to set the expectation that there will be an explanation behind the results of whatever you're testing. It's important to remember to plan ahead when you create a test, and think about explaining why the test turned out the way it did when the results come in.

Level up: Moving from a good to great hypothesis

Understanding what makes an idea worth testing is necessary for your optimization team.

If your tests are based on random ideas you googled or were suggested by a consultant, your testing process still has its training wheels on. Great hypotheses aren’t random. They’re based on rationale and aim for learning.

Hypotheses should be based on themes and analysis that show potential conversion barriers. At WiderFunnel, we call this investigation phase the “Explore Phase” where we use frameworks like the LIFT Model to understand the prospect's unique perspective. (You can read more on the the full optimization process here).

A well-founded hypothesis should also provide you with new, testable clues about your users regardless of whether or not the test wins, loses or yields inconclusive results. These new insights should inform future testing: a solid hypothesis can help you quickly separate worthwhile ideas from the rest when planning follow-up tests.

Ultimately, what matters most is that you have a hypothesis going into each experiment and you design each experiment to address that hypothesis.
– Nick So, Optimization Strategist, WiderFunnel

Here's a quick tip: If you're about to run a test that isn't going to tell you anything new about your users and their motivations, it's probably not worth investing your time in.

Let's take this opportunity to refer back to your original idea:

“My page needs a new CTA.”

Ok, but what now? To get actionable insights from 'a new CTA', you need to know why it behaved the way it did. You need to ask the right question.

To test the waters, maybe you changed the copy of the CTA button on your lead generation form from "Submit" to "Send demo request". If this change leads to an increase in conversions, it could mean that your users require more clarity about what their information is being used for.

That’s a potential insight.

Based on this insight, you could follow up with another test that adds copy around the CTA about next steps: what the user should anticipate after they have submitted their information. For example, will they be speaking to a specialist via email? Will something be waiting for them the next time they visit your site? You can test providing more information, and see if your users are interested in knowing it!

That's the cool thing about a good hypothesis: the results of the test, while important (of course) aren't the only component driving your future test ideas. The insights gleaned lead to further hypotheses and insights in a virtuous cycle.

It’s based on a science

The term 'hypothesis' probably isn't foreign to you. In fact, it may bring up memories of grade-school science class; it's a critical part of the scientific method.

The scientific method in testing follows a systematic routine that sets ideation up to predict the results of experiments via:

Collecting data and information through observation
Creating tentative descriptions of what is being observed
Forming hypotheses that predict different outcomes based on these observations
Testing your hypotheses
Analyzing the data, drawing conclusions and insights from the results

Don’t worry! Hypothesizing may seem ‘sciency’, but it doesn’t have to be complicated in practice.

Hypothesizing simply helps ensure the results from your tests are quantifiable, and is necessary if you want to understand how the results reflect the change made in your test. A strong hypothesis allows testers to use a structured approach in order to discover what works, why it works, how it works, where it works, and who it works on.

“My page needs a new CTA.” Is this idea in its current state clear enough to help you understand what works? Maybe. Why it works? No. Where it works? Maybe. Who it works on? No.

Your idea needs refining.

Let's pull back and take a broader look at the lead generation landing page we want to test.

Imagine the situation: you've been diligent in your data collection and you notice several recurrences of Clarity pain points – meaning that there are many unclear instances throughout the page's messaging.

Rather than focusing on the CTA right off the bat, it may be more beneficial to deal with the bigger clarity issue.

Now you’re starting to think about solving your prospects conversion barriers rather than just testing random ideas!

If you believe the overall page is unclear, your overarching theme of inquiry might be positioned as:

"Improving the clarity of the page will reduce confusion and improve [conversion goal]."

By testing a hypothesis that supports this clarity theme, you can gain confidence in the validity of it as an actionable marketing insight over time.

If the test results are negative: It may not be worth investigating this motivational barrier any further on this page. In this case, you could return to the data and look at the other motivational barriers that might be affecting user behavior.

If the test results are positive: You might want to continue to refine the clarity of the page's message with further testing.

Typically, a test will start with a broad idea — you identify the changes to make, predict how those changes will impact your conversion goal, and write it out as a broad theme as shown above. Then, repeated tests aimed at that theme will confirm or undermine the strength of the underlying insight.

Building hypotheses to create insights

You believe you’ve identified an overall problem on your landing page (there's a problem with clarity). Now you want to understand how individual elements contribute to the problem, and the effect these individual elements have on your users. It's game time – now you can start designing a hypothesis that will generate insights.

You believe your users need more clarity. You're ready to dig deeper to find out if that’s true!

If a specific question needs answering, you should structure your test to make a single change. This isolation might ask: "What element are users most sensitive to when it comes to the lack of clarity?" and “What changes do I believe will support increasing clarity?”

At this point, you'll want to boil down your overarching theme…

Improving the clarity of the page will reduce confusion and improve [conversion goal].

…into a quantifiable hypothesis that isolates key sections:

Changing the wording of this CTA to set expectations for users (from "submit" to "send demo request") will reduce confusion about the next steps in the funnel and improve order completions.

Does this answer what works? Yes: changing the wording on your CTA.

Does this answer why it works? Yes: reducing confusion about the next steps in the funnel.

Does this answer where it works? Yes: on this page, before the user enters this theoretical funnel.

Does this answer who it works on? No, this question demands another isolation. You might structure your hypothesis more like this:

Changing the wording of the CTA to set expectations for users (from "submit" to "send demo request") will reduce confusion for visitors coming from my email campaign about the next steps in the funnel and improve order completions.

Now we've got a clear hypothesis. And one worth testing!

Let's compare:

The original idea: "My page needs a new CTA."

Following the hypothesis structure: "A new CTA on my page will increase [conversion goal]"

The first test implied a problem with clarity, provides a potential theme: "Improving the clarity of the page will reduce confusion and improve [conversion goal]."

The potential clarity theme leads to a new hypothesis: "Changing the wording of the CTA to set expectations for users (from "submit" to "send demo request") will reduce confusion about the next steps in the funnel and improve order completions."

Final refined hypothesis: "Changing the wording of the CTA to set expectations for users (from "submit" to "send demo request") will reduce confusion for visitors coming from my email campaign about the next steps in the funnel and improve order completions."

Which test would you rather your team invest in?

Before you start your next test, take the time to do a proper analysis of the page you want to focus on. Do preliminary testing to define bigger issues, and use that information to refine and pinpoint your hypothesis to give you forward-looking insights.

Doing this will help you avoid time-wasting tests, and enable you to start getting some insights for your team to keep testing!