1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

1000+ Xenu hits in a week?

Discussion in 'Products & Tools' started by NewComputer, Mar 4, 2005.

0
  1. #1
    I checked my logs today, 1000 hits by Xenu which I thought I locked out with the following robots.txt:

    User-agent: xenu's Link Sleuth 1.1c
    Disallow: /

    User-agent: xenu's
    Disallow: /

    User-agent: xenu's Link Sleuth 1.2f
    Disallow: /

    User-agent: xenus Link Sleuth 1.1c
    Disallow: /

    User-agent: xenus
    Disallow: /

    User-agent: xenus Link Sleuth 1.2f
    Disallow: /

    User-agent: xenu Link Sleuth 1.1c
    Disallow: /

    User-agent: xenu
    Disallow: /

    User-agent: xenu Link Sleuth 1.2f
    Disallow: /

    I don't understand. I guess I can block via IP, but....
     
    NewComputer, Mar 4, 2005 IP
  2. Smyrl

    Smyrl Tomato Republic Staff

    Messages:
    13,740
    Likes Received:
    1,702
    Best Answers:
    78
    Trophy Points:
    510
    #2
    What is someone doing running Xenu on your site?

    That would worry me.

    Shannon
     
    Smyrl, Mar 4, 2005 IP
  3. NewComputer

    NewComputer Well-Known Member

    Messages:
    2,021
    Likes Received:
    68
    Best Answers:
    0
    Trophy Points:
    188
    #3
    Not to sure, I will have to look a little deeper...
     
    NewComputer, Mar 4, 2005 IP
  4. adamboalt

    adamboalt Peon

    Messages:
    5
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    whats zenu
     
    adamboalt, Mar 4, 2005 IP
  5. Smyrl

    Smyrl Tomato Republic Staff

    Messages:
    13,740
    Likes Received:
    1,702
    Best Answers:
    78
    Trophy Points:
    510
    #5
    Xenu is a link checker/site map builder.

    Shannon
     
    Smyrl, Mar 4, 2005 IP
  6. vagrant

    vagrant Peon

    Messages:
    2,284
    Likes Received:
    181
    Best Answers:
    0
    Trophy Points:
    0
    #6
    from their page here near the bottom


    • does not look like you can lock it out by robots.txt :eek:
     
    vagrant, Mar 4, 2005 IP
  7. flawebworks

    flawebworks Tech Services

    Messages:
    991
    Likes Received:
    36
    Best Answers:
    1
    Trophy Points:
    78
    #7
    This is what I believe is happening. I use xenu myself and I noticed it and I don't use it as much anymore. If you have check external links on; it checks the external links. So if you have coop ads running; say 5 on oh for giggles lets say 10,000 pages; it's gonna check those external links, on 10,000 pages if that's what you've selected. I don't think someone is doing it on purpose; they just don't realize what's happening.
     
    flawebworks, Mar 4, 2005 IP
  8. NewComputer

    NewComputer Well-Known Member

    Messages:
    2,021
    Likes Received:
    68
    Best Answers:
    0
    Trophy Points:
    188
    #8
    Thanks Flaw,

    I believe that is exactly what is happening. I tried to check the log files and could nto make sense of it, so that is what it must be.
     
    NewComputer, Mar 5, 2005 IP
  9. Smyrl

    Smyrl Tomato Republic Staff

    Messages:
    13,740
    Likes Received:
    1,702
    Best Answers:
    78
    Trophy Points:
    510
    #9
    That makes sense. Thanks for the information.

    S
     
    Smyrl, Mar 5, 2005 IP