Create a professional robots.txt file.

Generate a clean robots.txt file for your website by selecting crawler rules and adding your sitemap URL. Copy or download the generated file instantly.

Advertisement · AdSense-ready placement

Robots.txt Generator

Configure crawler access for your website.

User-agent Choose which crawler the rule applies to.

Sitemap URL Optional XML sitemap address.

Allow path Use / to allow the whole website or enter a specific path.

Disallow paths Enter one path per line. Leave empty when you do not want to block anything.

Common crawler rules

Block admin area Block private area Block internal search Block login pages

🔍

Control Crawlers

Define which parts of your website search engine crawlers can access.

🗺️

Add Your Sitemap

Include your XML sitemap location to make your website structure easier to discover.

⚡

Download Instantly

Generate the file directly in your browser and download it as robots.txt.