Robots.txt Generator

Search Engine Optimization

Robots.txt Generator


Default - All Robots are:  
    
Crawl-Delay:
    
Sitemap: (leave blank if you don't have) 
     
Search Robots: Google
  Google Image
  Google Mobile
  MSN Search
  Yahoo
  Yahoo MM
  Yahoo Blogs
  Ask/Teoma
  GigaBlast
  DMOZ Checker
  Nutch
  Alexa/Wayback
  Baidu
  Naver
  MSN PicSearch
   
Restricted Directories: The path is relative to root and must contain a trailing slash "/"
 
 
 
 
 
 
   



Now, Create 'robots.txt' file at your root directory. Copy above text and paste into the text file.


About Robots.txt Generator

Robots.txt In SEO

The file on which search engines bots take a look is Robots Exclusion Protocol Known as robots.txt file. This file tells the Search Engines crawlers that which directory is allowed to index and which is not. If you don’t make this file in your website, then there is a huge risk that The search Engine Crawlers may Index your full website pages or may not Index your website.

Why to use Robots.txt Generator by Super SEO Toolz?

A little mistake in robots.txt file may become a big disaster for your website. So we made this Tool for you to make a Robots.txt file for your website for free without any mistakes. Which help for your website Indexing in Search Engine without any problem.

 

Our Robots.txt Tool is free and also easy to use. 

How to use Robots.txt Generator by Super SEO Toolz?

There is not any rocket science in using Robots.txt generator by Super SEO Toolz and also it doesn’t require any knowledge to use it. You have to just follow these Simple Steps to Generate your own Robots.txt file for your website:-

  • When you go to Robots.txt Generator Page. You will see many options there, but not all options are required. But you have to choose it carefully to generate your Robots.txt without any mistakes. 

  • On the first Option Remain it as it is. It will allow all search Engine Robots to index your site.

  • In Crawl Delay Remain it as it is also. It will allow search engines to Crawl pages or posts without any delay.

  • On the Box next to it, Put your sitemap URL. If you don’t have Sitemap Use our Sitemap Generator To generate it for your website.

  • Then Next Steps is to choose which Search Engine Crawler bots you want to allow or disallow to index your website. If you want all Search Engine bots to index your site then don’t change it from Default.

  • After that enter the Restricted Directory that you want to Disallow the Search Engine Crawler robots to not index that.

  • Then Click Create Robots.txt to Generate Robots.txt. Or click Create and Save as Robots.txt, it will generate and make a downloadable robots.txt file for you.

Now Upload the file In your website directory.