Hello, I am new, and need some help on robots.txt

I found a robots.txt file on the root directory of my website, I did not create this file ( code below). My store and website is hosted in yahoo, Can anyone help me understand if this file is preventing all SE to index my pages ?
Thanks in advance for your help


User-agent: *
Disallow: /cgi-bin
Disallow: /RT
Disallow: /TG
Disallow: /QS
Disallow: /Img
Disallow: /I
Disallow: /wgl
Disallow: /P
Disallow: /ymix

# Rover is a bad dog <http://www.roverbot.com>
User-agent: Roverbot
Disallow: /

 

Hard to say. What is in all those directories on your site?

cgi-bin is the standard directory for scripts - usually disallowed
I'd guess that /Img is images - good to disallow

No idea what the rest are.

But if you didn't put it there, who did? Anyone else working on your site?

 

It is not disallowing Bots to Crawl Main site. just the script directory, Image directory and few other special directories. so nothing to worry. untill you want your images to appear in Google image search.
if you want your images to appear in Search, remove the line
Disallow: /Img
from the file and upload it to its previous location.

If this is Yahoo store site, Yahoo puts it automaticaly along with other scripts they put on the webserver.

 

everything seems of the norm so don't worry.

They will not hinder your results in the slightest.

 

I'm sure he's still worrying about this. Thanks Razor.

 

This code is generated by the Yahoo! engine, disallowing crawling over parts of the site irrelevant for SEs (checkout pages, for example). The relevant parts of the site will indeed be crawled with these definitions.

Tal Kedar
Research

 

we cannot edit the Yahoo store robots.txt, this is CLEARLY mentioned on yahoostore help page.

http://help.yahoo.com/help/us/store/promote/mail/mail-42.html

 

I noticed another site, topiccraze.com that has roverbot disallowed in their robots.txt with the following:

Any idea why these websites hate roverbot so much? More to the point does it make sense to disallow it from my site?

 

What I know about Roverbot, it is basically Email Gathering Robot...So every website disallow this robot...

 

Yes, it collects emails from websites

 

when ever i use to login in gmail 1 robot.txt file pops up.

i don't whats that .....

plz give some suggestion

 

I think ill disallow the rover bot after reading this.

 

Rover has been a bad dog indeed!

 

It's normal Yahoo robots file

 

plz edit all robot.txt file and create New file

User-agent: *
Disallow: /

I think it is best option for you..............