Content360 – is much more than an ordinary website crawler

Content 360 is designed to help you make informed and actionable decisions about your and your competitors’ websites and specific pages.

It allows you to crawl domains with a view to comparing competitor sites for structure, content, while identifying and fixing technical SEO and content errors.

Crawl your site and competitors

Integrates crawl, content, analytics and keyword data

Customise the crawler

Content 360 offers unparalleled insights into the performance of your campaigns’ content, providing one central location where you can access all the data needed to measure the success of your content, whether across one page, a group of pages or an entire website.

Site Crawler Analytics

For hardcore technical SEO audits, Authoritas’ powerful site crawler can help you identify broken links and duplicate content. Find what’s broken, and fix it fast.

  • Crawl your website and competitors
  • Discover what’s broken; links, duplicate content, server responses and more
  • Learn what’s working on your sites and competitors
  • Benchmark and compare SEO landing pages, internal links, backlinks, shares, and rankings

Content 360 can answer many questions about your own website and your competitors’ sites.

Website spider screenshot
SEO Issues on a website

Find SEO Issues Fast

Use the in-built filters to find site-wide issues quickly:

  • 301 Permanent Redirect
  • 302 Temporary Redirect
  • 403 Page Forbidden
  • 404 Page Not Found
  • 500 Server Error
  • 503 Unavailable
  • Canonical (External)
  • Canonical (Non-self)
  • Duplicated Descriptions
  • Duplicated Page Titles
  • Excluded By robots.txt
  • External Broken Links
  • Internal Broken Links
  • Long Page Titles
  • Missing Descriptions
  • Missing Headings
  • Missing Page Titles
  • No Index Set
  • Short Page Titles
  • Top 3 Ranks
  • Top 10 Ranks
  • Top 20 Ranks

Content & Crawl Data

By default, we will crawl up to 20,000 pages per domain.  If you need more pages ask about our Total Site Crawl service which can crawl potentially million-page websites.

You can display and easily export data for each URL such as:

  • Tier (#Clicks from Home Page)
  • Internal Links
  • Internal Anchor Text
  • Out Links
  • External Links
  • Organic Visits
  • Tags
  • Mapped Pages (to see related keyword rankings)
  • Advanced search operators to filter data
  • Page groups

Tip: Use the quick Show me” Filters help speed up the insights you can act on; 

  • Pages on this domain that rank for the keywords being rank tracked
  • Authored pages that rank well
Spider custom option selector
Website crawler integrated with backlink data

Offsite SEO

    • Are pages you’d like to perform better being shared and linked to as much as equivalent pages on the same topic for your competitors?
    • Which authors write content on a specific topic and how well is their content linked to and shared?

Architectural/Technical SEO

    • Is your website held back by technical issues such as duplicate titles, 404’s and broken links that need fixing?
    • Do the pages you’re trying to rank have a sufficient number of internal links and optimal link anchors to facilitate a high-ranking position for the targeted keywords?
Internal Link anchor text analysis
Popular website content analysis

Content Strategy

    • How popular is your content compared to similar content from a competitor?
    • How much content (content depth) do you have on a specific topic, and is there an opportunity to create more?

Content Performance

  • Which content performs the best in terms of internal links, channel traffic, rankings, backlinks and content engagement?
  • How much content (content depth) do you have on a specific topic, and how does that content perform?
SEO automated page performance analysis
Advanced search operators for our crawler

Insight at URL Level

Use multi-channel analytics, backlinks, rankings, and crawl data combined and search by keyword topic and by site area, e.g., Intitle:, inurl:, inpage: and more to find and analyse the right content fast. 

For each page you investigate you can see a complete 360 degree view.

  • Google Analytics Visits and Visitors
    • Direct
    • Organic Search
    • Referral
    • Paid Search
    • Social
  • Engagement Metrics
    • Bounce Rate by channel
    • Average Time on Page by channel
  • Optimization
    • $Value of the page
    • Mapped Keyword Phrases
    • Mapped Words
  • Page Performance
    • Top 3
    • Top 10
    • Top 20
    • Mapped Keywords
    • Mapped & Ranked Keywords
    • Mapped & Not Ranked Keywords
    • Mapped Potential Organic Clicks
    • Mapped Potential Media Value
  • Links
    • Internal Links
    • Outbound Links
    • External Links
    • Link Relevance
    • Top Internal Link Anchors
    • Top External Link Anchors

Website Crawling Customisation

You can create a crawl schedule for your site and even crawl your competitors’ sites, crawling up to a maximum of 20,000 pages per crawl.  

  • Daily
  • Weekly
  • Monthly
  • Recur on a specific day
  • Weekdays only

Advanced crawling options include:

  • Crawl start page
  • Crawl path
  • Ignore paths
  • Ignore parameters
  • User agent
  • Only crawl www
  • Start crawl in https
  • Obey robots.txt
  • Obey NoFollow attributes
  • Check external links
  • Check images
  • A specific text string to detect on each page
Website crawler custom settings

Enterprise option

If you need to crawl hundreds of thousands or even millions of pages then ask your Client Success Manager about the Total Site Crawl service.

“Authoritas offers a complete solution for creating effective SEO strategies. Their enterprise-level “big data” market analysis, combined with many tools for on-the-job SEO management mean they are the platform of choice for us. Their account management and responsiveness is also first rate.”

Ben Foster

Ben Foster

Managing Director, SEO Works

FAQs - Website Crawling & Content Optimisation

Can we help?

If you are interested in trying these SEO tools for your sites, then please book a brief call with one of our platform experts to explore whether we have what you need.

Research domain, page and keyword level SEO data