Hei I want to ask about my robot.txt Www.okecinta.com/robot.txt Is it good or not? Help me to improve it And,is it okay to use same robot on two sites?
You can check you robots.txt with Google Webmaster Tools - On the Dashboard, click the URL for the site you want. - Click Tools, and then click Analyze robots.txt. You can use the same robots.txt file for multiple websites, as long as you have the same directory structure and names, and want the same directories and files excluded. To improve it - You could always add the following to the end of robots.txt to help spiders find your sitemap. Sitemap: http://www.okecinta.com/sitemap.xml Code (markup):
I made a mistake in my earlier reply. I looked at the robots.txt file on your site, not the robot.txt file. You have both. The robots.txt file (with the s) is fine, however it blocks all robots from searching your entire website. Is that what you wanted? The robot.txt file (without the s) Will probably cause some bots errors with the * wildcard when used for directories and filenames. Be sure to spell it correctly. It must be robots.txt, and not robot.txt.
Oh my god,i just know if i had 2 robot. So,where is the correct one?and can u make me a new robot? Thx
Thx So i just need to rename robot.txt to robots.txt and add dissalow cgi? Is it okey if i use it for another blog?
Are your sure that the SEs support wildcard commands. Google is the only one that is talking about it.