What type of Robots.txt file allow all bot to visit my site, my blog and make index it? I want to know that what extension helps to allow all bot to index our sites and crawl them. thanks
A robots.txt file will simply tell a bot which files to access and index on your site and which not to, although the bot can simply ignore these directives if it decides. This file does not in itself tell the robot to come to your site and index it.
Thanks for your advice... I want to index my blog on Google but don't know how can I do it? please help me