robots.txt correct or not?

Discussion in 'robots.txt' started by jubcheng, Jul 7, 2008.

0
  1. #1
    Hi.

    Can someone of you guys tell me if this robots.txt file is correct or not?

    User-Agent: *
Disallow: /contact/form.php
Disallow: /contact/form.swf
Disallow: /contact/form.xml
Disallow: /contact/message_sent.html
Disallow: /contact/Scripts/
Disallow: /pdf/
Disallow: /styles/
Disallow: /time_script/
Disallow: /temp/
Allow: /
    Code (markup):
    If yes, how long does it take to remove the directories and files from the Google index?

    Any help is greatly appreciated.

    Thanks in advance,

    Jubcheng
     
    jubcheng, Jul 7, 2008 IP
  2. MOG

    MOG Well-Known Member

    Messages:
    2,219
    Likes Received:
    163
    Best Answers:
    0
    Trophy Points:
    180
    #2
    Hi JC,

    the robots you detail is fine - good job!

    now just make sure its at yourdomain.com/robots.txt and the pages will start to fall from google. It takes a few months generally as google need to re-spider each page thats indexed then realise its now out of bounds, then they drop it.

    if you want rid of the content in a much quicker fashion, submit a de-indexing request via google webmaster tools, that normall takes 2-3 weeks.

    good luck!

    MOG
     
    MOG, Jul 8, 2008 IP
  3. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #3
    It is absolutely fine..Good job dear..:)
     
    manish.chauhan, Jul 10, 2008 IP
  4. jubcheng

    jubcheng Member

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    36
    #4
    .......thanks a lot for both of your replies.......
     
    jubcheng, Jul 19, 2008 IP
  5. ASPkit

    ASPkit Peon

    Messages:
    53
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Looks good. Nice work. Google can take some time so be patient!
     
    ASPkit, Jul 19, 2008 IP
  6. catanich

    catanich Peon

    Messages:
    1,921
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #6
    I would remove the "Allow:" command, it is not needed

    User-Agent: *
    Disallow: /contact/form.php
    Disallow: /contact/form.swf
    Disallow: /contact/form.xml
    Disallow: /contact/message_sent.html
    Disallow: /contact/Scripts/
    Disallow: /pdf/
    Disallow: /styles/
    Disallow: /time_script/
    Disallow: /temp/​


    As to the delete all of these from Google, you must use the Google Webmaster Tools application. It requires site verification first (security issue)
     
    catanich, Aug 21, 2008 IP