We tested 1 Million Websites to see how Mobile-Friendly they are [Original Study]

Everyone is talking about the mobile-first world. The web as we know it is turning upside down.

All because people are spending a lot more time on their mobile phones.

Google is rolling out the mobile-first index. Voice search, AMP, PWA, Instant Articles, etc., are taking over the headlines.

All this fuss gives me the feeling that we forget something fundamental. Oh yeah, mobile friendliness.

Here is the punchline we worked a bunch to find: almost 24% of the top 1 million most popular websites in the world are not mobile friendly.

We used Google’s Mobile-Friendly Test to check Alexa‘s top 1 million websites on the web.

The first question that pops into my mind is: How can we talk about highly technical stuff when 1 in 4 websites is not even mobile friendly?

Let’s keep this in mind!

Mobile friendly vs. mobile first

We should make an essential distinction before we dig into the data we gathered on mobile friendliness!

A lot of designers/developers pop the question: why not responsive? What’s the difference between mobile-first and mobile-friendly?

First of all, we will see in the data that we can’t do responsive either.

Secondly, mobile-first is a mindset. Here are a couple of visuals explaining the difference:

(Image source)

(Image source)

The difference is subtle, but it’s there. I believe that once we get our heads around it, we will discover a whole new perspective.

Now let’s have a look at the data we gathered.

Score distribution

These are the scores for all the websites we analyzed:

Interesting findings:

• the minimum score to pass the Google Mobile-Friendly Test is 80%
• only about 17% of websites scored at 100/100
• the most frequent score is 99/100 (~25% of websites have this rating)
• the most frequent score for non-mobile friendly websites is 67% (more than 23K websites have this rating)

There are a handful of websites with scores under 50% which are not very visible in our main chart; zooming in:

Common errors

Here is the list of warnings we encountered for our websites:

As we can see, the most common warning is the “Links too close together”. It’s present among more than 77% of websites.

But how do these compare to Passed vs. Failed?

It would be interesting to see if a particular warning has a higher or lower impact on the mobile friendliness.

Voila:

Discoveries:

• links too close together doesn’t look like a complete deal-breaker
• if your text is too small to read there’s a 95% chance your website is not mobile friendly
• similarly, if your mobile viewport is not set there’s a ~88% chance you will fail the Mobile-Friendly Test (what is viewport?)
• content wider than the screen is sometimes acceptable (~54% chance to fail)
• even incompatible plugins (like Flash, Java, Silverlight, etc.) are acceptable sometimes (websites like Quora.com, Zillow.com or udemy.com are using incompatible plugins and are still passing the test)

Blocked and failed resources

There has been a lot of discussion on this topic as well. We should be aware that blocking access to CSS/JS/Image files inside robots.txt is a bad idea.

Basically, Google won’t be able to properly render your content.

But how are things in the real world?

If we were to look at the following graph without thinking things through, it would seem like a robots.txt apocalypse:

But if we compare to how many are failing the mobile-friendly test, we get this:

What we see:

• 60% of websites are loading resources that are blocked for bots; whoa!
• 34% of websites have resources that failed to load in our tests (errors, slow loading ones, etc.)
• bottom line: blocked/failed resources don’t seem to have too much of an impact on mobile-friendliness

Popularity

The mobile-friendly to non-friendly ratio is almost constant across Alexa ranks:

Being famous doesn’t seem to help with mobile-friendliness.

False negatives

Let’s have a look at aliexpress.com (NOT mobile friendly):

At first, I thought it was some kind of a mistake. Aliexpress is one of the most mobile-focused companies in the world.

After digging in a bit, I found out that they have a mobile version at m.aliexpress.com.

They just made a mistake on the rel canonical tag. Google clearly states that “m.” websites should point their canonical to the “www.” version.

In this study, we’re working within the limitations of Google’s tool, which is based on their rules.

Google is not God; they just have some guidelines.

If we don’t respect them, we might lose on the SEO front. But that doesn’t mean they own the absolute truth.

Methodology

We pretty much explained our research process throughout the article, but here the key points:

• we grabbed the top 1 million websites from Alexa
• we used Python to work with Google’s API
• we used R to clean and merge the data
• we used Tableau to visualize the results
• there are potential holes in the study
  • false positives like Aliexpress
  • about 100k websites returned errors/weren’t available for the Google test
  • blocked resources may cause false positives/negatives
  • working within the limitations of Google’s analysis
• here is our full list of results; have fun playing with it!
• here is the Tableau Dashboard with our study
• why Alexa? we just needed an extensive list of relevant websites

Bonus: mobile-first index live watcher

Most of us know Google said they are rolling out the mobile-first index. But smart kids like us don’t take their word for it.

We are looking at data!

So here is a live watcher from SEOLYZER that shows actual Googlebot data gathered from real-world access logs:

You can see the live mobile-first index watcher here.

I want to give a shoutout to Olivier Papon, the man who built SEOlyzer.io – a free tool for Log Analysis.

Even though, we like to make things complicated when it comes to log analysis for SEO, I must admit we are using SEOlyzer as well.

I will try to promote people who come close to our Growth Helping mentality with every chance I get!

Maybe some of you will follow.