How will I block Google Bot to arrive at my .gz sitemap

Discussion in 'robots.txt' started by e-marketing guru, Jun 30, 2008.

0
  1. #1
    I want to block googlebot to arrive at my .gz sitemap what will I have to do

    # Google
    User-agent: googlebot
    Disallow: /sitemap.xml.gz/

    Or,

    User-agent: Googlebot-sitemap.xml.gz
    Disallow: /*sitemap.xml.gz

    Or

    User-agent: Googlebot
    Disallow: /sitemap.xml.gz

    Or can I use like this

    # Google
    User-agent: googlebot (this will allow to visit all my website)
    Disallow:

    User-agent: Googlebot-sitemap.xml.gz
    Disallow: /sitemap.xml.gz

    May I use both of them together?

    Will this effect my overall crawling of Googlebot at whole website?
    Give me the honor to get a reply as soon as possible.
     
    e-marketing guru, Jun 30, 2008 IP
  2. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #2
    I want to block googlebot to arrive at my .gz sitemap what will I have to do

    # Google
    User-agent: googlebot
    Disallow:

    # Google
    User-agent: googlebot
    Disallow: /*sitemap.xml.gz

    Answer found in Google webmaster robots.txt analyzer.

    Now the question is should I block at .gz sitemap at MSN as well?
     
    e-marketing guru, Jun 30, 2008 IP
  3. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #3
    What exactly you want to do?? why you are blocking your sitemap from crawlers??
     
    manish.chauhan, Jul 1, 2008 IP
  4. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #4
    I am blocking Google crawler to crawl my .gz sitemap my xml sitemap is already in indexing.
     
    e-marketing guru, Jul 1, 2008 IP
  5. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #5
    e-marketing guru, Jul 2, 2008 IP
  6. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #6
    So why not you just block your /sitemap.xml.gz file from your robots.txt and remove this from indexing using URL removal in your webmaster tools..:)
     
    manish.chauhan, Jul 4, 2008 IP
  7. e-marketing guru

    e-marketing guru Active Member

    Messages:
    228
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    53
    #7
    will do this this time. So do u think the way I am using isn't fine???
     
    e-marketing guru, Jul 6, 2008 IP
  8. catanich

    catanich Peon

    Messages:
    1,921
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #8
    First, you can have both versions of Sitemap and not get busted. So blocking them is silly.

    But if you still want to, then use:

    User-agent: googlebot
    Disallow: /sitemap.xml.gz​
     
    catanich, Aug 21, 2008 IP
  9. zed

    zed Member

    Messages:
    57
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    43
    #9
    does it works for vbulletin
     
    zed, Aug 29, 2008 IP