1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Log Analyser to track spiders

Discussion in 'Traffic Analysis' started by flak, Sep 13, 2004.

0
  1. #1
    Anyone know of a good (preferably free or reasonably priced) log analyser that will track spiders effectively.

    I just want to know when a spider last visited and what file it requested on each visit. Ive been doing this with an access database but its awfully slow to process text files.
     
    flak, Sep 13, 2004 IP
  2. bobafind

    bobafind Peon

    Messages:
    128
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #2
    http://www.robotstats.com/en/

    I've had mixed results with this free script. It is sorta klunky, but better than nothing, and it keeps a log in a MySQL database so you can see historic data.

    If you want to see what it looks like:

    www.bobafind.com/robotstats/
    Code (markup):
    Obviously I don't want a live link there :)

    Also, hunting around I found this:

    http://www.darrinward.com/googletrax/

    The demo site is:

    http://www.darrinward.com/spydertrax/spyderview.cgi

    Make sure you select a date on the drop down list - I didn't and I thought that only basic information was listed. Once you select a specific date, it has a lot more details like Date / Time, User Agent, Crawler Remote Host, IP Address, Page.

    I haven't used it yet but I want to test it out.
     
    bobafind, Sep 13, 2004 IP
  3. Scoreboard

    Scoreboard Peon

    Messages:
    21
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    AWStats is great...and free.
     
    Scoreboard, Sep 13, 2004 IP