Tackling Duplicate Content on Mobile Websites

Midget runs away from cops

Mobile Browsers Are Smaller.

Today I want to talk about how to optimise your website for mobile browsers.
With the ever increasing use of mobile devices for accessing the internet, it would be foolish to overlook how your website performs on mobile browsers. There are many things to take into account when considering how best to present the content of your website to mobile users, not least that the screen size is drastically smaller, and navigation is often touchscreen based.

Some of the main things that need to be taken into account are that any navigation must be large and well spaced, and text should be presented in a large, clear format.

The Subdomain Solution (Bad)

Many websites have chosen to deal with the issue by detecting the user-agent of the users browser and sending the mobile visitors to a specific mobile subdomain (typically “m.domain.com”) where the information is presented in a format more fitting for mobile browsers.
The sad thing about this approach is that it really goes against the idea of Duplicate Content where it has been established that any given piece of content should have one single URL.
To address this problem we could do a few things such as canonicalising the pages to the URL of the page that is not on the mobile subdomain.
For example:

m.domain.com/article.html
is the mobile version of
www.domain.com/article.html

It would make sense to set the mobile version of the article as NOINDEX and add the following canoncial tag in the head of the page:
<link rel=”canonical” href=”http://www.domain.com/article.html” />
In this case we’ve made efforts to preserve any link equity that is pointed to the mobile version of the site, with the canonical tag, and prevent the mobile version from being listed in the Google search results.

Perhaps an easier move would be to block the entire subdomain in the robots.txt file of your website. Provided this was done before Google crawled the subdomain it would prevent the mobile version of your website from posing any duplicate content issues.
This would be the easiest to implement, however in this case you loose all link equity that is pointed at the mobile version. Sadly, neither of these options seem to be employed by most websites, including many big media companies.

For example:
Reuters
BBC
The Guardian

If we take a look at the BBC, for example, taking the first result we find in the m.bbc.co.uk subdomain:
http://m.bbc.co.uk/news/10519300
and compare it with the non mobile version:
http://www.bbc.co.uk/news/10519300

We can clearly see that both pages contain identical content.
The mobile version is not set as “NOINDEX”
No canonical tags are presented on the mobile page
The non mobile version does, thankfully, have a canonical tag :)
(This opens a whole new question as to why Google is indexing pages with duplicate content from the same domain when it is already aware of a canonicalised version, but that is a story for another day ;) )

Wait, are you saying the BBC has a huge duplicate content problem?
Potentially, Yes.
Currently totaling “About 7,910 results” according to Google.
Reuters comes in a little higher at “About 805,000 results”.

If we take some of the content of the Reuters article and search it on Google we can clearly see that both versions of the website are shown in the results:

Reuters duplicate content

So as we have established, this approach is something of a cluster-fuck when it comes to duplicate content. But what is perhaps even less forgivable is how this affects user experience.

For example, if I browse an article on my smartphone and decided to share it, either on a social network or by email, then the shared URL will be that of the mobile sub-domain.

Lets say I am keeping abreast of the latest Japanese blowfish news on my smartphone. I choose to share this article with someone and send them the URL I am currently reading: http://mobile.reuters.com/article/oddlyEnoughNews/idUSBRE83202P20120403

Go ahead and visit that link from your non mobile device.
What you get is the mobile optimised content filling your full-size desktop browser!

What you would need to do here is check the user-agent of each visitor to the mobile subdomain then redirect them to the full version if they are not using a mobile device.

This approach would also theoretically fix any duplicate content issues because the googlebot would always be redirected to the full version of the content if it tried to access the mobile version.
In this case link equity would also be preserved, for the most part, by a 301 redirect.

But still, this still feels a little over-engineered and where possible I always shy away from using 301 redirects. Many SEO’s regard 301 redirects as the silver bullet that cures a multitude of ailments. If you look at best practice for 301 redirects they should be used only when content has previously existed on a URL and now permanently exists on a new URL. In addition, adding another redirect to the pile makes it more likely that your URL will go through a bunch of redirects before ending up at the right place.
For example:

m.domain.com/article/abc/ >>>
m.domain.com/article/abc (loose the trailing slash) >>>
www.domain.com/article/abc (non mobile version) >>>
www.domain.com/article/abc-updated (updated version of the article)

Ok, so this is a bit of a “worse case scenario” example but by no means unimaginable.
At each redirect, the link equity passed forward to the article is reduced somewhat, you piss off the googlebot sending it through a chain of redirects, tie up more server resources, and overall it just looks unprofessional and dumb.

Surely we can do this a little better.

The Dynamic CSS Solution (Better!)

Surely a better solution would be to show both mobile and non-mobile visitors the same URL.

iphone website

danclarkie.co.uk as seen on an iPhone.


If we can do this then we automatically fix any duplicate content problems, we retain full link equity of any links pointing to the mobile version, and save putting another bloody 301 redirect out there in the universe.

Seems to make sense to me :)

So here’s what I suggest.

Obviously we want to present mobile friendly content to mobile browsers and still present the full power content to non mobile browsers, but we can easily do this without sending them to different URLs.

The simple plan is:

1: Detect the User-Agent
2: Deliver the relevant CSS file.
3: Profit.

This way we retain exactly the same source code for the page (aside from the different CSS file name) and deliver our content under one single URL, optimised for whichever type of device you are using.
Great success!

Firstly we will need to make a CSS file that is optimised for mobile devices. Some things to bear in mind here are, again, the reduced size of the mobile screen and the touch-screen navigation.

It is usually a good idea to make the font very big and well spaced, this can be achieved with the CSS attributes line-height and font-size
For example:

#content, #content input, #content textarea {
color: #333;
line-height: 40px;
font-size: 40px;
}

You may also need to hide a bunch of stuff that causes display errors, for this we can use the trusty old “display:none;” attribute. Whilst this tag always starts to upset some lesser informed ultra white hat SEOs, when used properly, as in this case, it presents no risk. Remember this CSS is only shown to mobile browsers and one can very easily justify that you are hiding certain, less important, attributes in the interest of user experience and making your content more accessible.

There isn’t really a better way to do this than working with a smartphone and testing changes in the CSS live on the device, tweaking it until it looks perfect.

Now you should have 2 CSS files, one for mobile browsers, and your original, non mobile optimised, CSS file.
If we go ahead and upload them to the server, naming them something like “style.css” and “mstyle.css” then we are almost done.

Next we still need to perform a little user-agent detection at the top of each page. If you are using WordPress or a similar CMS then this can easily be added to the header file of your current template.

<link rel="stylesheet" type="text/css" media="all" href="<?php 
if (preg_match('/(up.browser|up.link|mmp|symbian|smartphone|midp|wap|phone|android)/i', strtolower($_SERVER['HTTP_USER_AGENT']))) {
echo "**LINK TO MOBILE CSS FILE**";
}
else{
echo "**LINK TO NON MOBILE CSS FILE**";
} 
?>" />

What we’re doing here is checking the user-agent on the fly and delivering the correct CSS file to them based on their device.

Test it out on your smartphone and computer, if everything has worked you should see the mobile optimised page on your smartphone and the classic page on your computer.

No need to set up a bunch of complex canonical tags, NOINDEX tags, 301 redirects, etc.

We’ve instantly made your content much easier to consume on mobile devices without creating any duplicate content issues or affecting the style of your site for non mobile users.

Above all else, delivering good content in a way that is best for your users should be the main focus of any SEO effort ;)

Until next time, Happy Easter :)

3 thoughts on “Tackling Duplicate Content on Mobile Websites

  1. Pete

    Hi Dan, great article, agree that it makes a lot more sense to stick to the “one web” principle wherever possible.

    Just one question… If you’ve gone to the effort of building your website with identical markup but one set of CSS rules for mobile and another for desktop, rather than user-agent detection to direct the user to the correct version, why wouldn’t you use media queries to direct the user based on the width of the device they’re using?

    I thought this would be preferable over relying on a detection script that would need regular checking every time a new device / OS came on the market?

    Thanks again for sharing your knowledge, appreciate any feedback you can give on this.

    Cheers,
    Pete

    Reply
    1. danclarkie Post author

      Hi Pete,
      You’re right, you can achieve the same results with media queries. I’m just more of a PHP guy and did it that way. I guess by taking the user-agent you can be more specific and deliver content optimised to each individual device, if you so wished.
      However the theory remains the same, that you alter the CSS to the device and keep the HTML/URL identical :)
      I really should brush up on the use of media queries.

      Reply
  2. Ben

    What about if you want to offer the user a choice of mobile or desktop?

    Imagine they are using an iPad or tablet, some may want to choose, especially since a mobile site can save on data usage when using an iPad with 3G.

    Also, an iPhone is as capable as many desktops at viewing full websites, even without zooming. I often view a full website with my iPhone turned on its side.

    How would you suggest offering the user a choice (or a way to override the detected choice)?

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>