How to use the Robots.txt Generator
Fill in the fields above and the result appears straight away. Everything runs in your browser, so nothing you type is uploaded.
How it works
Outputs robots.txt directives from user-supplied values.
Examples
- With the default values
User-agent: * Allow: / Disallow: /private/ Disallow: /admin/
Limitations and assumptions
Robots.txt is a crawl directive, not access control. Never rely on it to protect private content.
Frequently asked questions
What does the Robots.txt Generator do?
Generate a simple robots.txt file with optional disallow paths and sitemap URL.
How does the Robots.txt Generator work?
Fill in the fields above and the result appears straight away. Everything runs in your browser, so nothing you type is uploaded.
What should I keep in mind when using the Robots.txt Generator?
Robots.txt is a crawl directive, not access control. Never rely on it to protect private content.
Can I see an example of the Robots.txt Generator?
Creates a basic user-agent block, allow rule, disallow entries and optional sitemap directive.
Is the Robots.txt Generator free to use?
Yes. It is free, needs no account and runs in your browser.