By combining insights from Google Adwords and Google Analytics, even during the first month of our optimisation, we achieved a monster 27% increase in their ecommerce sales (year on year).

Read the full case study here, you can’t argue with the numbers. Our Jenny really knows her stuff. If you fancy taking a look at some impressive cleaning machines, take a look at Bissell’s website http://www.bisselldirect.co.uk

As you can see from the photographs below this event has all the glitz and glamour associated with motorsport:

We had a fantastic day rubbing shoulders with some big names in motorsport.  It’s very interesting to see how motorsport is so data driven especially from a Conversion Rate Optimisation perspective.

Huge thanks to everyone at AutoSport for a great day out!

There is another kind of testing that is super useful and should be used just as much as MVT and A/B testing. It’s cheap, quick, flexible and extremely powerful.

Consider User Testing Videos.

What it is

There is a wide choice of companies on the Internet (a non-exhaustive selection is shown below to get you started on your search for the right partner) who provide this service:

• usertesting.com
• whatusersdo.com
• loop11.com

This is the kind of process you can expect to follow:

• You write a script: a short task description of a user journey that you want the tester to perform.
• You ask a set of questions that ask the user their opinion regarding the task.
• You specify what type of user you want to do the test.
• The degree to which you can specify the type of user will vary depending on who you use to run the test. Typically choices may include:
  • Gender
  • Age
  • Location
  • Income
  • Interests
  • Web experience
• You specify how many tests to run.
• Submit and pay for the test.
• Fees vary but expect to pay between £30-£50 per test.

Now, after a short period (many tests we’ve run yield results in 24-48 hours) you’ll get an email with a link to a video. The user’s browser will be shown in the video. You’ll see what they click and type and when. The user will provide a commentary on what they do, why and their opinions. They’ll answer your questions during the video.

The results will entertain, inform, surprise and maybe even frustrate you but the value of these tests has to be experienced. You should engage user testing videos with a certain mindset and with a systematic approach as you would to other tests. Here are some guidelines to help you get the most out of user testing videos.

How and When to use it

Optimisation idea generation:

Where do optimisation ideas come from? Your web analytics data is a great source. What if that silver bullet just isn’t leaping out at you? User testing videos are a super source of inspiration. Real users can give you insights that you may never have considered.

Optimisation idea confirmation:

Everyone in your office has an opinion about the site. Which idea do you choose to test? Assuming you have two great ideas that have the same economic potential but you can’t decide which to invest in first? Real users will give you a clearer path to explore. The usefulness of user testing videos as tools to end political arguments is hard to explain sufficiently. They’re amazing at ending long arguments about what to test.

Competitive Intelligence:

Using qualitative as well as quantitative data for analysis is essential as is data regarding your competitors. Real users will tell you what they like about your site compared to your competitors.

Build your tests carefully:

When comparing your site with a competitor, run multiple tests. Get users to try your site first and then your competitor’s. Then get more tests in the reverse order.

Think very carefully about the scripts you write. Don’t be too prescriptive such that the user does exactly what your hypothesis suggests. Make sure the test script is tight enough but not rigid to the point of being a self fulfilling prophecy.

Rerun a test after optimisation:

Having run an A/B or MVT test, go back to the same user (ideally) and run the same script. Ask them their opinion. Have you hit the pain point? Is the user experience better?

Volume:

You need more than one test for sure but you don’t need the same volume of testers as for an A/B or MVT test. You’re not looking for statistical significance. That is still the domain of the A/B and MVT test. You’re looking for guidance on the right test to run. Get a decent spread of opinions: 5-10 tests would normally be about right but it’ll vary.

In real terms, there are some core, highly tangible uses for Google Analytics Real Time:

Realtime marketing:

• Social
• TV/Radio advertising
• Hot news – great for publishers!
• Time based offers

So, you’re effectively marketing in real time right now.  You’re tweeting, posting on FaceBook, ‘plussing’ on Google+ (still not sure of the best verb here) and sending emails.  Where previously you would wait up to 24 hours to see solid processed data – even top level metrics – regarding the response to your marketing efforts, now you can see the response…well, in real time. Such responsive data has tremendous benefits for testing and measurement.

The advantages extend into the offline world too.  Advertising on TV or Radio?  You know when your ads are seen and heard, so you can have real time reports to see the effects online of your advertising offline.

We have a premiership football (soccer for the non-UK readers!) team on our client base.  Seeing the ebb and flow of a game on the pitch is fascinating for fans.  Seeing the correlation between on-pitch and on-line activity is highly actionable for marketers.

Google Analytics debugging – see the pages tracking in real time!

A great use of Google Analytics Real Time for ‘non-muggles’ is the ability to confirm the correct function of newly implemented tracking tags.  Yeah, Live HTTP Headers or Firebug in Firefox are still handy to confirm the __utm.gif request happens but now you can connect pretty much all the dots to debug and confirm a new analytics implementation.  Neat.

Going beyond the standard reports

This is all great but there is a lot that is not shown and rather too much that is shown in the Google Analytics Real Time reports.  By this I mean there are key metrics such as devices and browsers, conversions and revenue that are not shown in the reports.  Furthermore, the reports are unfiltered.  URIs contain querystring parameters, trailing slashes and are not lower case (all standard filters).

Fear not!  There is a solution.  Well, we have a solution brewing anyway.  This is early days and admittedly slightly hacky but worth sharing right now – feedback is a gift…

The technique is essentially to use two tracking codes – one for standard reporting and one for real time reporting.  The real time reporting GATC can use virtual pageviews to inject a lot of missing content that you want to see and exclude the content that you don’t want to see.

The two screen shots below show a contrived example where two pages were requested and shown in the real time content report.  The reported URIs bear no relation to the actual page URI.  We’re using virtual page views (VPVs) to add semantic value to the real time reports:

The first page is a key landing page.  The real time tracking code is semi-intelligent:  It knows we’re particularly interested in tracking pageviews on this page so the VPV is prepended with ‘/realtime’ to provide a key on which we can filter the real time content report.  Not all pages are tracked thus but we can filter the report for the ones we’re interested in.  If a campaign has multiple landing pages, just filter the content report for pages containing ‘realtime’ and you see only what you need to see.

Okay, we can see traffic sources and keywords but the ‘device’ report is becoming super important for a lot of clients.  Tablet and mobile device specific campaigns are of major importance.  In the second example, the user agent and therefore the device can be discovered and so is injected into the VPV.  The ‘realtime’ stem remains so we can still filter on any number of facets in the VPV.

Show me the money!

What about conversions and money?  You’ve noticed by now that there are a number of facets injected into the VPV.  If key conversion pages are ‘real time tagged’ with a semi-intelligent parallel tracker, we can track conversion pages in the same way that we track landing pages.  Make them stand out and filterable in the real time content report.  ‘Sale’ is a good example of flagging a receipt/thank you page.  Append the sale value and current revenue figure for the day and you can start to build a handy picture of the economic performance of a campaign in real time.

Rough edges

This is not over!  We’re not finished.  We’d love to hear feedback, other approaches, ideas, feature requests, criticism and praise (maybe!).  There is every possibility that these metrics are on the Google Analytics Real Time feature roadmap so the detailed implementation may have a short life span but we maintain that realtime reporting requirements differ from traditional reporting and web analytics.  This solution has legs – it needs to mature.

Technically, this approach is not without it’s pitfalls.  Agreed – there is risk in making a mess of your tracking with dual site trackers (please be careful if you try this).  Some data in the reports is less precise but good enough data is better than no data for our real time reporting purposes, right?

We’re working on the idea and we’ll keep you posted on developments.

One last thing…

Hello to Robin Dickinson – hope you enjoyed Think marketing ;-)

[Google Analytics Premium is] Now being used by some of the world’s most well-known organizations, Premium increases the power and ease of Google Analytics for your business. All for one flat-rate annual fee.

So, it’s already in the wild. What does it mean from a technical perspective?  The big headline is the freshness of the data.  The sheer processing power used to deliver data that is a maximum of 4 hours old 98% of the time is serious.  This processing power also raised data limits – up to 1 billion gif hits per month and the ability to download high volume unsampled reports.

An intriguing development is the increase of the custom variable limit to 50.  FIFTY! Imagine the advanced segmentation possibilities – 50! (50 factorial = 3 x 1064) advanced segments downloaded in unsampled reports.  Many analysts may fear fragmented data although if you’re dealing with large volumes and need more than 5 custom variables then this may be a current pain point rather than a risk.  Who would need such reporting horsepower?  If you’re conducting large scale RFM (Recency, Frequency, Monetary Value – http://en.wikipedia.org/wiki/RFM) analysis, the more fine grained you can make your attribute categories the better.

You may be wondering if these new features and capabilities require any change to tracking codes currently used?  No – and that is sweet.  Once you’ve signed up to use GA Premium – the change is made at Google’s end – not yours – until you need to go beyond 5 custom variables.

The feature list for Google Analytics Premium shows this is an enterprise scale product.  The roadmap should also make interesting reading but we’re not all privy to such sensitive information!   Premium is highly complimentary to the Standard Edition – both are extremely capable and powerful tools that compliment savvy Web Analysts and data driven marketers.  The future looks bright for the Google Analytics product suite.

Have you ever watched a movie based on a trailer only to find the 30 second trailer was a carefully crafted highlight and the rest of the movie was rubbish?

Have friends recommended restaurants only to find that the food and service were atrocious?

It really sucks to have your high expectations dashed on the rocks of disappointment and be left on a ship-wreck of disillusionment.  The same strangled metaphor goes for websites.

Consider a user arriving on your site from a Google/Yahoo/Bing search.  Say you rank highly for certain keywords related to your products or services:

• What if your landing page doesn’t deliver the promise of the finely crafted and optimised search result?

• What if your product range is broad and your homepage can’t cram all your awesome widget pictures on the page?

The user is going to be disappointed.  They’ll leave your site.  Sure, they might buy something else (one day) but you know that feeling of not finding just what you were looking for.

What if this didn’t happen and you could meet the user’s needs and exepctations every time they searched for your site, products or services?  You can. You just need to make your homepage (or any landing page) keyword aware.

What is it?

A keyword aware homepage (lets say all your landing pages are effectively homepages for now) will know when a user arrives on your site from an organic search; it will know what the user searched for and if it has some specific content related to the search, it’ll show it.

How does it work?

Your site can easily see when someone landed on the site from a search engine and what the user searched for – use the Google Analytics utmz cookie for an easy way to extract this information.  You could roll your own if you’re not using Google Analytics (GA) but as GA has done all the hard work in knowing what all the search engines are and how they send the search phrase used by the user, you may as well get GA do all the hard work for you!

So, you’ve captured the moment when the user lands on the site from a search and you know what the search phrase was.  What do you do next?

Choose keywords

Use a decent body of data from GA or Web Analytics tool of choice (up to 6 months should do the job nicely) to find the top 50 most common search queries used by users to find your site through organic search.  You could also use your top converting paid traffic search phrases too.

Use this data to understand user intent.  They are telling you what they want to find on your site through a neat little user intent survey tool called a search engine!  This is golden data – use volume and conversion rates (per search phrase) to find the hot topics that yield economic value on the site.

Okay, you know what users want when they search for your site, now there is one more step tp join up the dots.

Choose your content (carefully)

Here’s a suggested approach:

Use the most popular landing page (from organic search) for your first investment in this technique.  We’ll assume it’s your homepage and that you display a ‘hero image’ on the homepage.  Use javascript to determine when a user lands on the homepage from an organic search, find the search phrase and see if it’s in your magic list.  It is?  Great!  Now we need a hero image that matches the search phrase.  Is the user searching for the ‘turbo mega eagle thrust 2 pen’?  Right, you’ve got a high quality image of the product?  Super, show that image.

Rinse and repeat for more search phrases and images.

Caveat:
Choose the content you change based on keyword matches carefully.  Ensure the change in content does not get you slammed by search engines for sneakily changing content on the fly…you have been warned!

What if you can’t match the keyword to an image?  Show a default.  Simple.  Finished?  Nope – the next steps are equally important.

Measure the results

You’ve made a pretty neat change to the behaviour of your homepage.  You’ve made an investment.  As a serious data driven marketer you want to demonstrate to your HiPPOs that there is a significant return from this investment.  So you have to measure the effects of the keyword sensitivity.

We chose to record a session level custom variable in Google Analytics when keywords were matched and non-standard images were shown.  We recorded both the keyword and the image shown.  The power of custom variables is in using them to create advanced segments to see what behaviour the segment of ‘keyword matched’ users exhibited as opposed to those who were not ‘keyword matched’.  Over time you should see how users respond.

I’ll show you my findings later.

Give the keyword sensitive homepage a decent crack of the whip.  Let it run for a month and then take a look at the data trends.  How do users bounce?  Do they spend longer on the site?  Do they convert more?  Do they buy the products you’re showing on the homepage that match their search query?  Find the differences.  Build on the positives – make it work harder.

Optimise and scale

So, have you found the search queries that when matched reduce bounce rate and increase conversion rate?

Cool!  You’ve tackled your ‘head’ terms.  Ever heard of the ‘long tail’?  You need to tackle those guys next.  How long is your long tail?  Pretty long I’d guess.  So, you’re going to find a lot of terms that (individually) offer marginal gains.  Aggregate those marginal gains across a thousand keywords and if each term matched offers 0.1% improvement in bounce rate then…well you do the math(s).  It’s worth it.

Be advised, not all search terms will work well first time out.  You might have to change the image.  You might have to remove it from your list and just let the default hero image show – if it works better.  The advice here is to never stop optimising and testing new images and keywords.

Whilst optimising for keywords and images, maintain the performance of the match and display functionality.  Don’t let 10000 keywords in your list slow the display of the hero image down to a crawl.  As you scale up the functionality, make sure the speed scales also.

An example

All this is fine in theory.  How about some empirical evidence to support the hypothesis?

We ran a month long test on two sites: RadissonEdwardian.co.uk and TheMayFairHotel.co.uk:

RadissonEdwardian.co.uk with no matched keywords:

RadissonEdwardian.co.uk with ‘Heathrow’ matched keyword:

TheMayFairHotel.co.uk with no matched keywords:

TheMayFairHotel.co.uk with ‘spa’ match keyword:

Both sites were subject to the process described above and the results were extremely positive:

RadissonEdwardian.co.uk:

For users who had a search keyword matched:

• Pages/visit increased by 80%
• Time on site increased by 150%
• Bounce rate reduced by 66%

One keyword in particular stood out as ‘a real doozy’ (a very good thing):

• 98% increase in pages/visit
• 160% increase in time on site
• 76% reduction in bounce rate

TheMayFairHotel.co.uk:

For users who had a search keyword matched:

• Pages/visit increased by 20%
• Time on site increased by 30%
• Bounce rate reduced by 18%

Again an outstanding performer keyword is worth highlighting:

• 43% increase in pages/visit
• 58% increase in time on site
• 51% reduction in bounce rate

Conclusions

The results above show the potential of the technique.  These were results for head terms.  Realistic expectations for long tail terms would be less in terms of metric deltas but aggregated across a larger scale implementation the returns become more significant.

No SEO impact has been recorded during these studies due to careful choice of content and thorough testing of the implementation.

As we saw above, your results may vary.  The imagery shown on TheMayFairHotel.co.uk was already spectacular (as you would expect for a high end luxury London hotel) hence the degree of uplift was less but still strongly positive.

We’re now in the process of scaling this behaviour and await further significant results.  Watch this space.

In the meantime – can you find more keywords that yield the described behaviour on the sites above? ;-)

We have a lot of cool clients at ConversionWorks but Morphsuits take the prize for mixing fun with cool with great business practice.  With a terrific Facebook following and a fantastic, fun and easy to use site from our sister company WebExpectations Morphsuits has taken fun fashion and fancy dress to a new and exciting level.

It’s really great to see our clients in the press.  The guys were recently featured on BBC Scotland – take a look!

ConversionWorks is working with Morphsuits to continually optimise the Magento driven sites (currently 13 countries are supported with multilingual and multicurrency features).  We’re using Google Website Optimiser to continuously run A/B and Multivariate tests to optimise the site. We’re also running a series of paid advertising campaigns on AdWords, Yahoo, Bing and Facebook. With the business growing at a rapid rate, it’s clear the team have achieved a great mix of fun, fashion and smart business practices to rise to the challenge.

Both companies will be working with the ConversionWorks experts to help them improve the performance of their website analytics and improve their online conversion rates.

To see if we can help you increase your conversion rates call us on 0845 500 408.

And then you see something like this:

It looks like the test variations are going south, tanking, showing an #epicfail. Meh. How can this be?  The new designs were awesome.  Everyone loved them.  The designs addressed the issues and concerns outlined in the user testing.  The data clearly showed that users didn’t like the page.  The bounce rate was too high and the conversion rate was too low.

Ah ha!  There is the clue.  When you set up any test, you use testing software (lets assume it’s Google Website Optimiser (GWO) for now) to measure a chosen metric and the test software does a great job of doing this.  Thing is, there are other metrics that will be affected by the test.  If you measure click through rate on a page, your test designs will likely affect conversion rate and bounce rate among other metrics. GWO doesn’t measure the changes in these other metrics.   Best practice dictates that you couple Google Analytics (or similar web analytics tool) to your test so you can measure all your metrics on a ‘per test variation‘ basis.  This then lets you look beyond the primary test metric and helps you fully understand the economic value that each test variation offers.

Taking the example above (this is a real scenario we experienced recently), the original page exhibited a higher CTR than the test variations.  This was with good reason.  Users were confused as to which option they should click on.  So they clicked, stayed confused and clicked back to click again.  Users clicking on test variations were less confused and so clicked less…but then converted more!

We looked at the goal conversion rate in Google Analytics for the original and test variations and saw the following results:

Double ah ha!  #epicwin!  This confirmed the hypothesis regarding the user behaviour.  Despite the primary test metric failing, the overall economic value was confirmed.

Lessons to learn from this case study

• Always integrate Google Analytics (or other web analytics system) when testing.
• Measure micro-conversions in the test software
• Measure macro-conversions in web analytics

It would have been so easy to flag this test result as a failure when analysing the test result in isolation.  Taking other metrics into account enabled the correct test winner to be chosen.  Which is the correct test winner?  Well, do you notice how the second variation in the test results was the best converting variation? If you analyse test results according to your original hypothesis, this will help you think about more than one metric and to consider economic value: which test variation delivered the best economic value?  This might differ from the primary test metric but the aim of testing is to maximise economic value in the end.

New Variation:

Old Variation:

Not everyone is seeing this variation though – according to our Google Account Manager, just 1% of traffic across Google.com and local domains as part of a global calibration experiment.

As far as paid ads go, this complete repositioning of the display URL follows a number of changes that have already been rolled out this year – first the wrapping of headlines with ad text, then the appearance of display URL domains alongside headlines. This latest change however, would apply to all ads rather than just those eligible to appear in the top box.

Whilst the new layout looks cleaner and more spacious, the increase in white space means that more scrolling up and down the page is required to view all ads. This surely can’t be good for CTR – one of the main determining factors of user engagement? So it will be interesting to see if the changes get pushed live across all searches.

Interested to hear the thoughts of anyone else who’s seen this variation…