Is this robots.txt blocking Slurp ?

[Percept]

Hi,

I got the following robots.txt file from a robots.txt generatoir:


# All robots will spider the domain
User-agent: *
Disallow:

# Disallow directory /admanager/
User-agent: *
Disallow: /admanager

# Disallow directory /wp-admin/
User-agent: *
Disallow: /wp-admin

# Disallow directory /wp-includes/
User-agent: *
Disallow: /wp-includes

# Disallow directory /mag/
User-agent: *
Disallow: /mag


Google has thousands of pages indexed, so does MSN but Yahoo falls behind with only 133 pages in the last week. Could it have something to do with this robots.txt ?

[Jim bob 9 pants]

have a look here ROBOT.TXT VALIDATION (http://www.searchengineworld.com/cgi-bin/robotcheck.cgi)

Worth a try - dont think you need User-agent: * each time though

Jamie

[just-4-teens]

here is all you need

User-agent: *
Disallow: /admanager
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /mag

the above the allows search robots to index the entire site exept from the directories listed.

[mussolinihitler]

i would recommend what "just-for-teens" had said too. Why do you have to make this robots.txt so long ??