I Need a Better Server Log Analyser - Suggestions Please

[spidermonkey]

Thanks to Shawns excellent Keyword Tracker putting me back to where I used to be in terms of interrogating Google, my webmaster toolset is almost complete. The bent spanner in the box is my server log analysis software. The program I have at the moment does a good job with live traffic - the problem comes when it archives the data as it doesn't archive the query string. Experienced Google optimisers will know that Google ignores what's in the keywords meta tag and draws its own inferences from the page as to appropriate search terms to refer for. Very often you will look at the query string attached to a Google referral and see a combination of terms that you didn't optimise for. It would be a shame to make changes to that page to optimise it for terms you hope Google will refer for - only to de-structure it so you lose the referrals that Google definitely was providing. So you need to be able to look at the query strings historically. I have a workaround in the form of server-based visitor tracking that does a great job of allowing me to historically analyse queries, both for referrer and destination pages, but it has no real export facility - so I have to copy and paste if I want to load the terms into a spreadsheet - or indeed Keyword Tracker. I would like to be able to do everything from within the same environment and export data when needed.

The short version of all this is that I need an off-line log file analyser, that I can download server logs to, that archives the query string and has good export facilities (at the very least as .csv files). It needs to run on a windows PC.

Any suggestions would be really helpful.

Thanks in anticipation. :)

Mike

[hans]

i have one provided by host - i seldom use it

here my current week online:
http://www.kriyayoga.com/wwwstat/index.html

but i use and prefer webalizer - OFFLINE or online working
homepage
http://www.mrunix.net/webalizer/

search strings can be configured for all SEs

also stats by country, entry-/exit-pages, kB, hits, hours, users, etc

many things can be configured easily

its fast and easy for huge access_log files !! its written in C , free
can be run online and offline available for win and ix platforms ( inux, etc )

here is a sample online output from webalizer
http://www.mrunix.net/webalizer/sample/usage_199905.html


and the following Google query string should give you some real online stats created with current version

http://www.google.com/search?client=googlet&q=Usage%20Statistics%20Generated%20by%20Webalizer%20Version%202.01

query string used above ( if above URL broken ) is:

Usage Statistics Generated by Webalizer Version 2.01

---> this is default text appearing on all output pages :-) unless otherwise configured

[rustybrick]

urchin.com

[spidermonkey]

Thanks Guys. :)

I already have server-based analysis. Pay per click ROI analysis is for other projects. I need off-line server log file analysis where you FTP your server logs down to your computer and analyse them off-line.

Mike

:D

[rustybrick]

ClickTracks.com :)

[hans]

webalizer is just that - a pure and very fast offline - logfile processor

( that also can be used online )

the reason for the online links i gave you is to show you what kind of output webalizer gives you. ( can NOT send you my oflfine stat by mail as sample - its about 120+ MB !)

here is another online output of such stats from webalizer from a real site in italy ( a friend of mine ) - you get ecxactly the very same output ...

when you ftp your logfiles and process them all offline !

http://www.filmscoop.it/stats/
( btw this domain has been rejected by Google for G ads !! he applied about a week ago or so .. its all about film review - some 5+Mio hits/m ... ) but NO from G ..

--->>
when you click on the MONTHS of the stats - then you get the full details ..

[Owlcroft]

Is anyone familiar with Analog 5? My ISP gives me a choice of that or Webalizer.

[hans]

have a look at BOTH URL i gave above in an earlier post above

... at the VERY bottom of MY ISP provided log ( URL somewhere above in a previous post )

This analysis was produced by analog 5.22. ...

i have never used analog 5.22 offline - ( have no idea if that is possible ) i have no further knowledge except the online stats i see for my own site .. since years
and the offline webalizer i use since years ..
webalizer output see above other URL

[expat]

Off line - Quick and fast one with nice trend analysis etc

http://www.weblogexpert.com/

I use the lite version for small sites / clients

For more info on or offline I use AWstats
M

[spidermonkey]

Thanks everyone for the info - I'll start going through the specs :D

Mike

[spidermonkey]

I should have mentioned at the start of the thread that I need to analyse IIS enhanced format log files so the powerful, yet free Webalyzer is out. ClickTracks is the tool for the job if I were running extensive pay per click campaigns in addition to regular analysis, but is a bit pricey for me (seeing as I use pay per click more for filling in the gaps). Weblogexpert looks like the tool for me so far. I don't begrudge them 80 dollars for the full version. Logfile analysis is too important not to be prepared to pay some money for it. If it was in my budget I would choose Clicktracks Pro.

Many thanks :)

Mike

[Daniel]

WEBALIZER alllll the way :p its what we use to provide stats to our customers.

[apondonet]

The combination of these two has produced fabulous results for me:

>> Weblog Expert
>> Statcounter

Do a Google search - Apparently, I am not yet allowed to post live links !!

Pay special attention to Statcounter ....It rocks !!

[hans]

while i have zero knowledge about your "IIS enhanced format log files"
i am at 1and1 hosting since august this year and they too have a modified access_log
hence i asked a C developper to make a patch for the modified log format
a matter of minutes and webalizer works fine again.
may be same is possible for IIS enhanced format log files as well - depending on how much your format is different from apache log files
the patch was / is most likely needed for ONE file only all else original webalizer.

may be that also helps
http://www.geocities.com/danilody/webalizer/windows.html
or another google search ..

[flawebworks]

I should have mentioned at the start of the thread that I need to analyse IIS enhanced format log files so the powerful, yet free Webalyzer is out.

Your host can tweak this individually in IIS for you; but any info logged before that won't be recorded; unless you want to do it manually.

[NewComputer]

I love weblogexpert... I was a log time user of WebTrends but the price is just too out there. I tried the demo version of WebLogExpert and have been a user since. Missing a few things but plenty good enough for log analysis away from the server.

[J.D.]

> I need an off-line log file analyser, that I can download server
> logs to, that archives the query string and has good export
> facilities (at the very least as .csv files). It needs to run on
> a windows PC.

I'm using Stone Steps Webalizer (http://www.stonesteps.ca/webalizer/). It is really fast, ~400 log rec/sec without DNS lookups. It generates very customizable, but static (eg. no sorting) HTML reports. It is possible to export individual entries (eg. user names, IPs) as tab-delimited files, but there seems to be no way to export report data.

> I should have mentioned at the start of the thread that I need
> to analyse IIS enhanced format log files so the powerful,
> yet free Webalyzer is out.

This version does handle IIS log files.

J.D.