How to prevent crawling from yahoo.

Discussion in 'Yahoo' started by neerajk, Nov 30, 2007.

0
  1. #1
    Hello,

    I want to prevent my site to be crawled from yahoo.
    for this I inserted

    User-agent: *
    Disallow:

    User-agent: Slurp
    Disallow: /

    in robots.txt

    but it did'nt work

    Please tell me the solution.

    Regards
     
    neerajk, Nov 30, 2007 IP
  2. neelesh

    neelesh Peon

    Messages:
    141
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Any particular reason for stopping yahoo to crawl your website?
     
    neelesh, Nov 30, 2007 IP
  3. iaria

    iaria Peon

    Messages:
    202
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #3
    iaria, Dec 1, 2007 IP
  4. Divisive Cottonwood

    Divisive Cottonwood Peon

    Messages:
    1,674
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I can't think of any reason why you would want to stop Yahoo crawling your site :confused:
     
    Divisive Cottonwood, Dec 1, 2007 IP
  5. Robert T

    Robert T Peon

    Messages:
    105
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Not sure why you would want to do this either.
    Reverse the two entries and slurp will go away.

    User-agent: Slurp
    Disallow: /

    User-agent: *
    Disallow:
     
    Robert T, Dec 1, 2007 IP
  6. unknownpray

    unknownpray Active Member

    Messages:
    3,831
    Likes Received:
    14
    Best Answers:
    0
    Trophy Points:
    70
    #6
    Yahoo is now working along with Bing so I don't understand why you would want to prevent Yahoo from crawling your site. Yahoo is the second largest search engine being used after Google so if you want to prevent this from crawling then you might be saying good bye to a large number of users.
     
    unknownpray, Sep 19, 2011 IP
  7. uday nikam

    uday nikam Peon

    Messages:
    318
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #7
    unknowpray is write why are you decide this because crawling is helped to your website, and maximum people are use this to optimize any search.
     
    uday nikam, Sep 23, 2011 IP