Anything wrong with robots.txt?

Discussion in 'Google Sitemaps' started by bryanw, Mar 24, 2008.

  1. #1
    hello,

    I have a site, the main page is '/index.php'. And I have a blog '/blog' made by wordpress.

    Here is my robots.txt,

    User-agent: *
    Disallow: /wp-
    Disallow: /wp-admin
    Disallow: /wp-content
    Disallow: /wp-includes
    Disallow: /comments/
    Disallow: /trackback/
    Disallow: /feed/
    Disallow: /comments/feed/
    Disallow: /trackback/
    Disallow: /*.aspx
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.gz$
    Disallow: /*.wmv$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$

    Disallow: /blog/wp-
    Disallow: /blog/wp-admin
    Disallow: /blog/wp-content
    Disallow: /blog/wp-includes
    Disallow: /blog/comments/
    Disallow: /blog/trackback/
    Disallow: /blog/feed/
    Disallow: /blog/comments/feed/
    Disallow: /blog/trackback/
    Disallow: /blog/*.aspx
    Disallow: /blog/*.js$
    Disallow: /blog/*.inc$
    Disallow: /blog/*.css$
    Disallow: /blog/*.gz$
    Disallow: /blog/*.wmv$
    Disallow: /blog/*.cgi$
    Disallow: /blog/*.xhtml$

    Sitemap: /blog/sitemap.xml


    Thanks!
     
    bryanw, Mar 24, 2008 IP