robots.txt question

google webmaster said that the url on my site http://theguitarresource.com/forums/index.php is restricted by my robots.txt file. is that because it ends in .php? should i remove this from the robots, i'de assume my forum is a good page to be indexed...

User-agent: *
Disallow: /wp-content/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /cgi-bin/


User-agent: Googlebot
# disallow all files ending with these extensions
Disallow: /*.php$
Disallow: /*.js$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /*.gz$
Disallow: /*.wmv$
Disallow: /*.cgi$
Disallow: /*.xhtml$

# allow google image bot to search all images
User-agent: Googlebot-Image
Disallow:
Allow: /*

# allow adsense bot on entire site
User-agent: Mediapartners-Google*
Disallow:
Allow: /*

 

why are u hiding .php file from google? there's nothing for it index.

 

also i did a search for my indexed pages and just this came up: theguitarresource.com/wp-admin/edit.php?paged=2

first of all why is that the only page indexed??? 2nd, shouldn't this be blocked by my robots file?? thanks!

 

your robots is completely f'd. just use this:

 

Allow *.php , it will help you.

 

just block admin and image folders. allow rest of files and folders.

 

is this better?

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-


# allow google image bot to search all images
User-agent: Googlebot-Image
Disallow:
Allow: /*

# allow adsense bot on entire site
User-agent: Mediapartners-Google*
Disallow:
Allow: /*

 

Hi,

User-agent: *
Disallow: /wp-

Code (markup):

is probably good enough!

It allows all robots everywhere except in areas with a URL starting with /wp-.

When you give a robot special instructions with a directive like
User-agent: Googlebot
you instruct this robot to only look at the instruction following this line and to disregard the instructions following
User-agent: *

Hope this helps.

Jean-Luc