Robots.txt Generator
Create a robots.txt file to guide search engine bots on how to crawl your website.
Ad Advertisement (Top Banner)
Default - All Robots are:
Allowed
Refused
Crawl Delay (Seconds)
Default (No delay)
5 Seconds
10 Seconds
20 Seconds
Sitemap URL (Optional)
Restricted Directories (one per line)
Create Robots.txt
Your Robots.txt
Copy
Ad Advertisement (Bottom)
Ad Advertisement (Sidebar)