Plans
Platforms
Description
Parse your robots.txt file easily with our free online service. Submit the form, enter your robots.txt file, and the path you want to check. Get parsed results just like Google's crawlers.
User-agent: ia_archiver
Disallow: /dashboard/
User-agent: Googlebot
Disallow: /login/
User-agent: Exabot
Disallow: /private/
