The findings were shocking; most worryingly almost half of the 1,000 websites analysed were found to have either very basic SEO or in the worst cases absolutely none.
Purpose of the Study
To compete in business today, whether at a local, national or international level, a fully functional website is a business imperative.
Thomson Local has been helping customers find local suppliers for decades; now more than ever consumers are turning to search engines like Google and Bing, as well as local search websites like Thomson Local.com for help in finding the best local suppliers.
Whilst the increasing use by consumers and B2B buyers of online directories and search engines to find suppliers brings tremendous opportunities for well prepared businesses of all sizes; the spectre of increased competition from other local, national or international competitors is a serious threat to companies that have not adequately invested in establishing their internet presence.
Thomson Local and Analytics SEO were keen to establish how well the UK’s businesses were prepared for this increasingly competitive world of online competition.
It is hoped that this study can be conducted annually and perhaps even internationally, to benchmark the state of UK business websites against other industrialised nations.
The main goals were:
- To understand the level of investment made by UK businesses in optimising their sites for search engines
- To understand whether larger companies were better optimised than smaller companies
- To establish whether the length of time a business has been trading has a bearing on the quality of its website
- To assess whether companies were using website analytics software and statistics packages to measure the return on investment from their websites
Thomson Local is one of the UK’s leading local business search companies and has an up-to-date database of over 2.2 million actively trading businesses. The Thomson Local Database was the ideal source for this type of analysis, as unlike other public sources of company information like Companies House which record dormant business records and holding companies; the Thomson Local Database is updated continuously and only contains details of actively trading companies.
A representative sample of 1,001 businesses with websites were selected from the Thomson Local Database segmented by ‘employee size’ and ‘number of years in business’ to give a representative sample of all UK trading businesses.
Analytics SEO set out 30 criteria as key signals of SEO quality and fed them into their pioneering SEO software that was able to automatically and simultaneously perform an SEO audit of all 1,001 sites.
For each website analysed we looked at:
- Indexation – how well a site is indexed in each search engine and the local version of that search engine
- On-Site Optimization – known technical and content issues which have a positive impact on a website’s opportunity to rank well in search engines
- Off-Site Optimization – external links to the website from other websites; the quality and quantity of external links is known to have a positive impact on a site’s ability to rank well for its desired keywords in search engines
- Page Rank
- Pages indexed in Google
- Pages indexed in Yahoo
- Pages indexed in Bing
- Listed in DMOZ (Open Directory)
- Listed in Yahoo Directory
- Indexed in Google Local
- Indexed in Yahoo Local
- Indexed in Bing Local
- SEO-friendly URLs
- Basic analytics tracking code
- Advanced analytics tracking code
- Home page load time
- HTML sitemap
- XML sitemap
- Custom 404
- Domain age
- Pages crawled
- Descriptive page titles
- Descriptive page descriptions
- Unique page titles
- Unique page descriptions
- Images with missing alt tags
- Pipe character in page titles
- Site under construction
- Inbound links to deep pages
- Inbound link count
- Deep link ratio
Once the website audit was completed each website was awarded a score; the higher the score the better optimised for search engines the site was considered to be.
Validity of SEO Website Quality Scores
These scores were then correlated with the Google Page Rank (Google’s assessment of a site’s quality – see Wikipedia entry for more details) for each website to assess whether the results were in line with the distribution of Google Page Ranks. As you can see from the graph below, the distribution of the Analytics SEO site scores was sufficiently close to Page Rank to determine that the results were valid for the purposes of this study.
For ease of analysis these SEO scores have been categorised into 4 Levels of SEO: None, Basic, Intermediate, Advanced:
- None – Score <=25 – No SEO factors in place.
- Basic – Score <=50 – Only a few basic SEO factors in place
- Intermediate – Score <=90 – The major basic SEO factors are in place but there is still a need for fine tuning of on-site optimisation and greater external link building.
- Advanced – Score >90 – Represent the very best optimised sites. The majority of SEO factors are in place and the need for on-site optimisation is low. However, they may still benefit from further external link development and continued content development in order to compete.
Key Research – Summary of Findings
Here we share some of our key research findings. The bottom line is that the majority of UK businesses have poorly optimised websites.
The general state of UK business websites
- 47% of all business websites have none or very basic levels of SEO
- Over 70% of companies with less than 50 employees have none or very basic levels of SEO
- Unsurprisingly larger companies tend to have better websites
- With a few exceptions, the length of time a business has been trading or the type of business has little on no bearing on the quality of its website
- 12% of all business websites are not indexed in any search engine
- 0.6% of business websites are under construction
- Only 3% of all business websites could be considered to have advanced levels of SEO
Technical Configuration of Websites
- 74% of businesses have no basic website analytics software for tracking site visits
- Only 2% of all businesses used advanced website analytics software
- 82% have no HTML sitemap
- 75% have no XML sitemap
- 69% have no Robots.txt file configured
- 71% have no Custom 404 page configured
- 35% do not have unique Page Titles
- 56% do not have unique Page Descriptions
- The average company with less than 50 employees has 408 inbound links
- The average UK company has 6,874 inbound links
Clearly, a study like this generates a lot of paperwork and more charts and tables than we can publish on this blog. So here are a few of the most pertinent findings that relate to the search engine optimisation and website development marketplace.
SEO Site Score by Company Size
Company Size (by number of employees) is a good indicator of the likely quality of its website. (0.9 Correlation).
SEO Site Score by Length of Time in Business
The length of time a company has been trading is not a good indicator of the likely quality of its website.
Use of Website Analytics Software by Company Size
As one might reasonably expect, there is a strong correlation between the size of a business and the likelihood of it having website analytics software configured on its website.
Website analytics is a pre-requisite for all websites (whether for profit or not). How else can you establish whether the money and time you are spending on your website is fruitful or not.
N/B: Although our SEO software can track in the order of 200 different types of website analytics software, it is not possible to track server log based analytics packages and it may be that the vast majority of small businesses are using this to give them website visitor data. However, in our experience these packages do not offer the level of usability and reports that a small business owner really needs and we would still recommend that small business owners install Google Analytics or other free website software applications on their site.
Website Configuration – Basic Technical SEO Factors that help with SEO
Installing HTML and XML sitemaps and submitting the latter to the search engines is one of the simplest and most beneficial things that a website owner can get done to help get their website indexed properly initially and every time it is updated. Configuring the robots.txt file properly so the search engines know which content to index and which content not to index is also fundamental and recognised as best practice; as is setting up a 404 page which serves a friendly screen to anyone or any search engine bot which follows a link to a page on your site which has moved or is no longer there.
Yet, over 70% of UK companies with less than 50 employees do not have these basis fundamentals in place. Even, half of companies with less than 500 employees are yet to get this right.
Website Configuration – Basic SEO Content Factors that help with SEO
Having unique, relevant and descriptive meta data (page titles, descriptions, keywords, H1 tags and so on) as well as keyword rich URLs is the fundamental platform upon which a good website is built. Yet, with the exception of image alt tags, over 70% of all companies do not have these basis SEO fundamentals in placed.
Website Configuration – All SEO Technical & Content Factors
- When you put the last three graphs together you get an even clearer picture of the proportions of businesses that do not have these basic measures in place.
- Over 70% of UK companies with less than 50 employees need to get their sites configured properly for search engines. This is not an expensive option and something that should be within easy reach of all small businesses.
The irony is of course, that the vast majority of these business owners do not have a basic website analytics package installed, and it is therefore going to be difficult for them to work out if any investment in optimising their sites is going to pay off in terms of visitors or sales!
Content – Average No. of Pages Indexed in Google by Company Size
Unsurprisingly, larger companies have more pages indexed in Google; most likely because they have larger sites with more content; but also because they are more likely to use HTML and XML sitemaps to help the Googlebot index their content.
Sitemaps help get your content indexed
Sitemaps are a good way for companies to get more content indexed in Google. Sites with HTML and XML sitemaps on average have more pages indexed in Google.
Content Indexation by Search engine by Company Size
We also compared by company size the number of pages that were reported to be indexed in each search engine versus the number of pages our crawler found in the time allocated (24 hours per site).
The graph shows what you might expect, that larger sites will have more pages in each search engine. It also shows, unsurprisingly, that Google is much better at indexing the websites of bigger companies than our spider; for example, for companies with greater than 500 employees Google reported 263% more pages than we reported in 24 hours. Although, it is worth noting that Google may have built up its index over months not one day!
If there is such a thing as an average UK business~, then according to this study it has:
- 289 pages on its website (crawled by AnalyticsSEO)
- 267 pages indexed in Google
- 764 pages indexed in Yahoo
- 30 pages indexed in Bing
~ We recognise that averages are not tremendously helpful for this type of analysis.
It is interesting to note, across the board, that Yahoo seems to over report the number of pages indexed compared to our spider and Google. Although it was beyond the scope of this research to explore this further, our experience with other analysis indicates that this could be that Google does a better job than Yahoo of filtering out duplicate content.
How popular is the average website?
This graph shows what you might reasonably expect that larger companies tend to have sites that attract more links. The average number of inbound links and the percentage of deep links (links that go to pages other than the home page) rise as the company grows.
This could be for a whole host of reasons including; the size of their sites (larger companies have bigger websites) and the fact that their sites are likely to be indexed more fully; the quality of their content (too subjective to be measured) but it’s not unreasonable to expect that larger companies may have bigger budgets to spend on website content; and the size of their marketing budget and what proportion of this is allocated to online, link building and online PR.
A summary would indicate that:
- The average company with less than 50 employees has 408 inbound links~
- The average UK company has 6,874 inbound links~
~ Again we recognise that averages are not tremendously helpful for this type of analysis.
However, for SEO consultants and agencies, we hope that as a rough rule of thumb it may help you show your client that for a company of its size they really should have more inbound links. Clearly, it’s no substitute for sector level analysis of your competitors’ inbound links to see how competitive your sector is and what the gap is between you and the competition.
This research spells good news for the SEO industry in 2010 as it shows that there is clearly tremendous headroom for growth as the majority of websites still need SEO work.
But it does beg the question what the website design and development industry are doing! Ensuring that a website is well optimised for search engines and local online directories should be a fundamental pre-requisite for every new website project and website re-design project for that matter.
The website design industry is mainly comprised of a cottage industry of one-man-bands and small firms; and many businesses will have had their website built by a neighbour, friend, relative or potentially worse may have bought one of the poorly optimised website design templates available online. Clearly, standards across the board need to be raised and website design companies need to be using SEO software tools like Analytics SEO to ensure that the websites they build are up to scratch.
Further Information & SEO Research
We have developed unique automated search engine optimisation software available as SAAS (software as a service) which is currently being used by 4 out of the Top 10 UK SEO Agencies and hundreds of enterprise clients.
If you are interested in search engine optimisation analysis in bulk of hundreds or thousands of websites for market research, market analysis or other marketing purposes then please do get in touch.
If you are interested in building a user-friendly and search-engine friendly website then contact Thomson Local about its range of reasonably priced websites for SMEs.
If you have interest in conducting this research in another country please do get in touch.
By: Laurie OToole
CEO and founder of Authoritas. Entrepreneur, SEO and golfing enthusiast!