Do Yahoo Spiders Ignore No Index No Follow?

Discussion in 'Yahoo' started by shirsc2, May 14, 2007.

  1. #1
    Recently Yahoo has indexed many of my pages that have the no index no follow in the robots.txt file. These are all different landing pages that may be considered duplicate content which is why they were set up no index no file.

    Does Yahoo ignore this command? It doesn't make any sense to me.
     
    shirsc2, May 14, 2007 IP
  2. oseymour

    oseymour Well-Known Member

    Messages:
    3,960
    Likes Received:
    92
    Best Answers:
    0
    Trophy Points:
    135
    #2
    what exactly did you put in the robots file?....the no index and no follow belongs in the meta tags. you use dissalow in the robots file
     
    oseymour, May 15, 2007 IP
  3. M Yasir

    M Yasir Peon

    Messages:
    32
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    YES!

    I have experimented that if you use "nofollow" in url. Yahoo! slurp will ignore it and index your pages.
     
    M Yasir, May 16, 2007 IP
  4. axlarry

    axlarry Notable Member

    Messages:
    3,961
    Likes Received:
    121
    Best Answers:
    0
    Trophy Points:
    240
    #4
    Yes. I thought that belong to meta tags too. And I guess yahoo ignore this comment as I saw backlinks from SB sites and wiki to my sites.
     
    axlarry, May 17, 2007 IP
  5. reyotech

    reyotech Peon

    Messages:
    102
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I guess Yahoo doesn't really care lol,

    that's good for people who use blogs for links
     
    reyotech, May 17, 2007 IP
  6. uttoransen

    uttoransen Prominent Member

    Messages:
    8,227
    Likes Received:
    550
    Best Answers:
    0
    Trophy Points:
    320
    #6
    even google indexes links and pages from nofollow attribute, but then how much weight they give on those no follow links is yet unknown!:rolleyes:
     
    uttoransen, May 17, 2007 IP
  7. biz2bee

    biz2bee Peon

    Messages:
    98
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I read an article at SE Roundtable and as far as I can recall either MSN or Yahoo paid attention to the nofollow url tag. As for meta nofollow or disallow I agree with seo ranter - check out how and where you tagged it.
     
    biz2bee, May 17, 2007 IP
  8. gravtex

    gravtex Peon

    Messages:
    18
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    That's been the thing I've found interesting about nofollow links, the name implies that the spiders won't follow them, however, that hasn't been the case. While it doesn't appear (in my own testing at least) that Google gives them any weight, Yahoo certain seems to.

    I've noticed that when I tested blog commenting as a way to promote a site, my Yahoo & MSN traffic jumped up.
     
    gravtex, May 18, 2007 IP
  9. biz2bee

    biz2bee Peon

    Messages:
    98
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #9
    thats an interesting observation, I will keep that in mind :)
     
    biz2bee, May 18, 2007 IP
  10. trichnosis

    trichnosis Prominent Member

    Messages:
    13,785
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    300
    #10
    nofollow tag has no meaning for yahoo not. if there are pages that you dont want to be crawled by bots, you must add this pages to your robotx.txt
     
    trichnosis, May 24, 2007 IP