I love split testing - Visual Website Optimizer Blog

---

How RuneScape Leveled Up Revenue Through Process-Driven CRO Posted: 13 Oct 2016 05:03 AM PDT The following is a case study about how RuneScape followed a structured conversion optimization (CRO) program to increase revenue on its website. About RuneScape RuneScape is a fantasy massively multiplayer online role-playing game (MMORPG). It was developed by Jagex and launched in January 2001. The popularity of the game is enormous. RuneScape has welcomed over 250 million players to its world since its release. More than 2 million users play every month, and millions more watch avidly through social channels. RuneScape has consistently strived to deliver a great experience to its users—not just limited to the game but also on its website. After all, it's the website where users find forums and game guides, and buy in-game items. The CRO Team Rob Marfleet, UX Specialist at Jagex, takes care of User Experience and CRO across the payment flow on the website (the payment gateway and its preceding pages). Dave Parrott, Payments Services Director at Jagex, and Nastassja Gilmartin, Payments Manager at Jagex, help Rob in identifying testing opportunities and analyzing test results. Rob Marfleet, UX Specialist at Jagex, takes care of User Experience and CRO across the payment flow on the website. Rob works with teams of designers and developers that help facilitate implementation of winning test variants on the RuneScape website. Additionally, Rob takes help from Disha Ahuja, Client Success Manager at VWO, to utilize the VWO platform to its full potential. About the Case About 50% of users on the RuneScape website arrive as direct traffic. The other half of the traffic consists of users from referrals, social media, and email marketing campaigns. Rob adds, "This is mainly down to RuneScape enjoying a very loyal user base, with many players having played for several years." The CRO team aims to optimize high-potential pages, that is, pages that are closest to the payment gateway and require minimum effort in optimization. The Treasure Hunter page on the website is one such high-potential page that the team chose to optimize. The Treasure Hunter page lets users buy keys to unlock treasure chests in the game. The treasure chests contain items that can be used within RuneScape. Rob explains, "Treasure Hunter activity is an optional mini-game within RuneScape—keys are earned through play, but can also be gathered in bundles that are purchasable on the site." This is how the original page looked like: On clicking Continue on the Treasure Hunter page, users are directed to a Payment page where they can choose from multiple treasure chest packages. Payment page The RuneScape CRO team thoroughly analyzed the Treasure Hunter page and identified optimization opportunities. Next, the team used VWO to capitalize on these opportunities. Optimization Process The CRO team followed the following process to improve conversions on the RuneScape website: Setting a Goal Finding Opportunities for Optimization Creating Hypothesis Developing Variation Analyzing Test Results Setting a Goal The goal of the optimization campaign was to grow revenue by increasing the number of purchases. Finding Opportunities for Optimization The team at RuneScape studied a heatmap of the Treasure Hunter page. The heatmap showed that a significant number of users were clicking the Get Keys section on the page—a section which was not clickable. Users perhaps either wanted a direct access to the keys or wanted to search for further information. Heatmap of the original page Next, the team watched visitor recording sessions on the page and observed that a lot of visitors on the Payment page returned to the Treasure Hunter page. The team realized that the Treasure Hunter page probably did not offer sufficient information about the treasure chest packages to users. Creating Hypothesis The team hypothesized that providing details about treasure chest packages on the Treasure Hunter page will lead to greater conversions on the Payment page. Developing Variation Based on the hypothesis, the team created a variation of the Treasure Hunter page. The variation included a new section highlighting four treasure chest packages. Here's how it looked: An A/B test was run to find the better performing version between the original page and the variation. Analyzing Test Results The test ran for a month from August 15–September 13, 2016. The variation outperformed the control and increased the number of purchases by almost 10 percent. Test result report on VWO Rob shares his learning from the A/B test: "I think one of the more important aspects to take note of here is that the page variation actually resulted in less traffic to the payment page, but increased the amount of purchases made. Effectively, we can say pretty confidently that by giving the users package information upfront, we created higher quality traffic to the next stage, simply through transparency, and informed the user before going forward—users who went to the purchase page already knew what they were after. This is incredibly useful when considering other areas of the payment flow—if the effect can be replicated, it can potentially translate to more wins." Next Steps The CRO team did not stop after it found success with the A/B test. The team felt that the variation can be optimized even further. The team realized that the offer of four treasure chest packages can possibly leave the users spoiled for choice. The team hypothesized that recommending one of the packages to users will help them choose better and, consequently, increase conversions. Based on this hypothesis, the following variation was created: The variation featured a Recommended package. This variation was pitted against the winning page from the first A/B test. The variation won and further increased the number of purchases by almost 6%. Experience Using VWO Rob shares, "As a hands-on user of VWO, I've personally experienced how quickly it allows prototyping and testing of new ideas, features and content. The ability to push changes, without having to involve multiple teams to relaunch areas of the site can't be praised highly enough, and the ability to reverse those same changes instantaneously is equally as useful. It's allowed me to run a number of campaigns straight away that would normally have to be scheduled further down the line, at a more opportune moment, and that's pretty invaluable. Using the actual software is very straightforward and easy to understand—campaigns can be built in a short period of time, and having Disha available any time to help determine the best testing practices has definitely helped me find wins—she's super friendly and eager to help, and I've already implemented several testing campaigns that have been borne out of collaboration between her and myself, one of which, is in the process of being fully implemented on the site." What Do You Think? Do you have any recommendations on how RuneScape can further improve user experience and conversions on its website? Did you get any conversion optimization ideas for your own online enterprise? Tell us using the comments section below. 0 0 ratings How will you rate this content? Please choose a rating The post How RuneScape Leveled Up Revenue Through Process-Driven CRO appeared first on VWO Blog.

You are subscribed to email updates from VWO Blog. To stop receiving these emails, you may unsubscribe now.	Email delivery powered by Google
Google Inc., 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States