Will Google Listen to my robots text file like this?

Discussion in 'Search Engine Optimization' started by bbrian017, Jun 19, 2009.

0
  1. #1
    I’m wondering if Google will follow my robots text as is or do I need to do something special to it?

    http://www.blogengage.com/

    I added it into the root!

    # All robots will spider the domain
    User-agent: *
    Disallow: /templates/
    Disallow: /3rdparty/
    Disallow: /libs/
    Disallow: /modules/
    Disallow: /plugins/
    Disallow: /internal/
    Disallow: /backup/
    Disallow: /thickbox/
    Disallow: /api/
    Disallow: /evb/
    Disallow: /avatars/
    Disallow: /admin_index.php
    Disallow: /admin
    Disallow: /login.php
    Disallow: /js/
    Disallow: /img/
    Disallow: /upcoming.php
     
    bbrian017, Jun 19, 2009 IP
  2. karpok

    karpok Active Member

    Messages:
    325
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    60
    #2
    Yea, it is good.
    Google follows the instructions written in ur robots.
    And you are uploaded it in ur root.
    So, it works. No problem.
     
    karpok, Jun 19, 2009 IP
  3. bbrian017

    bbrian017 Well-Known Member

    Messages:
    2,990
    Likes Received:
    66
    Best Answers:
    0
    Trophy Points:
    170
    #3
    Thanks I never did one before!

    Due to the large amount of spam I don't' want Google to index upcoming.php apparently it's bad for my website!
     
    bbrian017, Jun 19, 2009 IP