Robots.txt file and indexed pages..

Discussion in 'Google' started by kes, Aug 1, 2007.

0
  1. #1
    Hello guys,

    I made the mistake of not using a robots.txt file for certain pages of my sites.

    The pages are in google, if i now use the robots.txt file to disallow the robots, will google de-index it ?

    or do i need to do this manually using google webmaster tools ?

    Cheers

    Kes
     
    kes, Aug 1, 2007 IP
  2. indyguidedotinfo

    indyguidedotinfo Notable Member

    Messages:
    3,254
    Likes Received:
    202
    Best Answers:
    0
    Trophy Points:
    245
    #2
    in webmaster tools there is an option to block URLS from being spidered. just add the url that that and google will stop spidering it and hopefully remove it.
     
    indyguidedotinfo, Aug 1, 2007 IP
  3. classifieds

    classifieds Sopchoppy Flash

    Messages:
    825
    Likes Received:
    51
    Best Answers:
    0
    Trophy Points:
    150
    #3
    I would do both - webmaster tools to make sure - and robots.txt to keep them from being indexed again.
     
    classifieds, Aug 1, 2007 IP
    Harry_Potter likes this.
  4. websys

    websys Active Member

    Messages:
    841
    Likes Received:
    23
    Best Answers:
    0
    Trophy Points:
    78
    #4
    websys, Aug 1, 2007 IP
  5. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #5
    if you add a page which is indexe by google to your robots.txt file, google will delete that page from the index
     
    trichnosis, Aug 1, 2007 IP