Sitewide optimisation split-testing for TheDungeons.com

It’s easy these days to split test the impact of a button colour, or even totally different page content, but what about testing changes across a whole site… that was the challenge on TheDungeons.com.

Services:

Split testing on a website is a lot of fun.  It’s a way to test two or more versions of a page to see which one ‘wins’ according to given criteria.  This could include bookings, revenue, conversion rate, sign ups etc.

Test, don’t guess - whatever we’re trying to improve this is what we always say at Semantic.  Data Drives Decisions.

It’s important to try and gather data and test in real world conditions before making updates, if at all possible.  

For TheDungeons.com, we wanted to validate that our new improved menu functionality was just that - an improvement.

But how do you go about testing an entire menu system on a live site?

That’s where server-side testing comes in...

What we did

We implemented code on TheDungeons.com to enable server-side testing.  This meant that we could test sweeping site-wide changes that would be hard to do with other tools.  

Without getting too technical, the process has a few key steps: 

  • Implement core codebases changes to handle traffic allocation
  • Hook up site templates with experiment changes
  • Enable CMS settings to control experiments across multiple sites around the world
  • Hooked into Google Optimize server side experiment API
  • Made Google Tag Manager changes for data collection
  • Tested across multiple Dungeon sites to ensure data validity
  • Deploy updates on all Dungeon sites

There’s a whole lot more to it than that, but once it’s setup, we can then test all sorts of sitewide changes: 

  • New headers or footers
  • New menu functionality
  • New designs shared across the site

The results then feed into Google Optimize, which also takes the Google Analytics data and uses this to report on which experimental variant is most successful.

"I have worked with the team from Semantic for more than 3 years for various brands. Very open-minded, constantly evolving attitude to improve our digital performance. A very hands-on mentality, easy to reach out to."

- Joerg Bergmann, eCommerce Manager Europe

The Results

The tools and techniques here helped us to run tests on TheDungeons.com over the course of 2019, which helped feed into their overall growth (see Growth on a Global Scale Case Study) 

Some of the things we were able to find out were: 

  • Are hover or click drop down menus the best user experience?
  • Should canonical pages be used, or is it best to link straight to the relevant page?

If you have similar questions and want to optimise your website then drop us a line, we’re always happy to help.