robots.txt and IP addresses

Discussion in 'robots.txt' started by Maciej Ziemczonek, Apr 8, 2008.

  1. #1
    Hi

    I have a problem. Our site is located on a couple of servers. Google is somehowe indexing also direct urls of these servers.

    Therefore, for example, we have in google:

    www.ourdomain.com

    and

    www.ourdomain.hostnameserver1.com
    www.ourdomain.hostnameserver2.com

    What's more, also IP addresses of these severs are indexed as seperate urls.

    We have only one robots.txt, the same meta tags are for the whole site. The structure of catalogues and files is the same for each server - which makes it impossible to block access to chosen catalogues and files.

    Do you have any idea how to:

    1. block Googlebot from crawling these urls?
    2. remove these urls from Google index?

    I'd appreciate you help greatly,

    best regards
    Maciej
     
    Maciej Ziemczonek, Apr 8, 2008 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
  3. Maciej Ziemczonek

    Maciej Ziemczonek Peon

    Messages:
    9
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Blocking and redirecting is one of the solutions we are considering. Thanks!
     
    Maciej Ziemczonek, Apr 8, 2008 IP