170 Spiders - Must see amazing

Discussion in 'Yahoo' started by evilopinions, Apr 7, 2008.

0
  1. #1
    evilopinions, Apr 7, 2008 IP
  2. GetSmart

    GetSmart Peon

    Messages:
    14
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    how do you detect that spiders are crawling your site and which spiders? i need help so i can get over this learning curve.
     
    GetSmart, Apr 7, 2008 IP
  3. kartal

    kartal Banned

    Messages:
    296
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3
    cool!

    btw,how can i see it on a wp site?
     
    kartal, Apr 7, 2008 IP
  4. AffiliateBot

    AffiliateBot Guest

    Messages:
    70
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Nah, that's nothing. Many big forums have hundreds of spiders from various search engines crawling their pages everyday.
     
    AffiliateBot, Apr 7, 2008 IP
  5. BruceW4yne

    BruceW4yne Member

    Messages:
    55
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    48
    #5
    Check your server log for user agents, or if you're on a cPanel server, click on "Latest Visitors," or "Awstats."
     
    BruceW4yne, Apr 7, 2008 IP
  6. Robert_M

    Robert_M Banned

    Messages:
    257
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Cool! Thanks for sharing! ;)
     
    Robert_M, Apr 7, 2008 IP
  7. dairyman

    dairyman Notable Member

    Messages:
    7,224
    Likes Received:
    274
    Best Answers:
    0
    Trophy Points:
    205
    #7
    With so many spiders going around the corners of your site, what happened to the bandwidth? Are you on a dedicated hosting plan or shared?
     
    dairyman, Apr 7, 2008 IP
  8. godsofchaos

    godsofchaos Peon

    Messages:
    2,595
    Likes Received:
    124
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Lol.. yeah saw a few similar threads already.... yahoo must be going Brooom broooom!
     
    godsofchaos, Apr 7, 2008 IP
  9. snowbird

    snowbird Notable Member

    Messages:
    3,036
    Likes Received:
    395
    Best Answers:
    0
    Trophy Points:
    290
    #9
    Maybe with all these bots crawling pages Yahoo will finally start listing deeper pages without a lot of seo work. Maybe I'm just dreaming and wasting bandwidth for nothing...
     
    snowbird, Apr 7, 2008 IP
  10. sonnekonige

    sonnekonige Banned

    Messages:
    15
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    My forums has more bots than it does people. That's good for indexing i think ;D
     
    sonnekonige, Apr 7, 2008 IP
  11. evilopinions

    evilopinions Guest

    Messages:
    69
    Likes Received:
    5
    Best Answers:
    0
    Trophy Points:
    0
    #11
    It ate atleast 1.8 Gig of bandwidth yesterday and I see a few pages indexed at yahoo if I search http://law9.com.

    However there has been no significant rise in traffic or anything due to this. There must be mods that actually tell you that there are spiders crawling...
     
    evilopinions, Apr 8, 2008 IP
  12. forumtyrone

    forumtyrone Peon

    Messages:
    577
    Likes Received:
    9
    Best Answers:
    0
    Trophy Points:
    0
    #12
    1.8GIG Bandwidth for a forum with very few posts? :rolleyes:
     
    forumtyrone, Apr 8, 2008 IP
  13. deejne

    deejne Well-Known Member

    Messages:
    53
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    101
    #13
    very interesting
     
    deejne, Apr 8, 2008 IP
  14. snowbird

    snowbird Notable Member

    Messages:
    3,036
    Likes Received:
    395
    Best Answers:
    0
    Trophy Points:
    290
    #14
    Yahoo is now sucking up 3 times the bandwidth that Google is using...
     
    snowbird, Apr 8, 2008 IP
  15. Masticate

    Masticate Peon

    Messages:
    39
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #15
    Sounds messy
     
    Masticate, Apr 8, 2008 IP
  16. lbalance

    lbalance Peon

    Messages:
    381
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    0
    #16
    ive noticed within the last month that there are always at least 20 yahoo spiders on my forum.
    last week it jumped up to 320 at one time.
     
    lbalance, Apr 8, 2008 IP
  17. machinegun87

    machinegun87 Peon

    Messages:
    580
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #17
    This is a cool image.

    No wonder some new forums with hardly >100 members sometime have "Most users online" above 500+.
    The spiders help to reach that figure I guess.:p
     
    machinegun87, Apr 8, 2008 IP
  18. pscoln1

    pscoln1 Peon

    Messages:
    31
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #18
    Do you usually have more spiders if your on a dedicated hosting plan?
     
    pscoln1, Apr 9, 2008 IP
  19. kingofsanda

    kingofsanda Peon

    Messages:
    8,154
    Likes Received:
    218
    Best Answers:
    0
    Trophy Points:
    0
    #19
    That's great I have never paid much attention to this on my sites, but I guess it's important.
     
    kingofsanda, Apr 9, 2008 IP