Analytics SEO tutorial video on the Robots.txt component.[youtube id=”qyMm3WJmho4″]
This component is another part of the system’s daily health check. It will simply try to locate a robots.txt file on your website. It is usually found off the main directory of your homepage.
The component explains why having a robots.txt file setup is important and will show you the content of the robots.txt file, if we have located one. Again, you can add notes here and mark the task status accordingly.
If you have any suggestions as to how we could improve this simple component, by all means click on the feedback tab here and send your idea into us. If we feel it will add even more value to the system, we will add it to our product roadmap.