Robots.txt & Sitemap Validator

The robots.txt and sitemap validator checks the two files search engines read before anything else on your site. It fetches robots.txt, validates every directive line by line, and points out the mistakes that quietly cost you indexing: a stray "Disallow: /", a 5xx response that makes Google pause crawling the whole site, rules written before the first User-agent. It then follows the Sitemap: declarations to the sitemaps themselves.

Every sitemap is checked, including all the files listed inside a sitemap index: well-formed XML, the correct namespace, entry counts and size limits. Each tag is explained separately — which ones are required, which ones actually influence crawling, and which ones (like priority and changefreq) both Google and Yandex state they ignore, so you can safely drop them. The first URLs from the sitemap are requested for real, so a sitemap full of 404s or redirects does not go unnoticed.