Robots.txt Generator: instructions and limitations
Write a simple robots.txt with selected allow and disallow paths and a sitemap address.
How to use Robots.txt Generator
Enter allowed and disallowed URL paths, a user-agent name and optional sitemap URL. Generate a minimal robots.txt.
What to know before you start
Robots.txt is a crawling instruction, not access control. Disallow never protects confidential pages, and a misplaced rule can affect search discovery.