Stitching Log Files together??

[carowan]

Is there a way to combine weekly log files into one large file, or at least link them sequentially?

I am currently using WebLog Lite, and it does a fine job with the weekly logs, but I am interested in seeing trends over time, (frequently used keywords, visitors etc) rather than a weekly snapshot. I realize that the files can be large, but im sure someone has figured out a solution for this way before I thought about it.

Also I have a large discrepancy (approx. 50%) between my log file and my webhosts (Yahoo) online tools in counting the total number of visitors. Which is generally more accurate?

Thanks

[exam]

Well no one has responded to this Q. I would say you can d/l the raw log files, and put the contents in one big file, and analyze it with a prog. (webalizer, analog) on your own computer. Should be no problem.
/d/

[mopacfan]

A good log file analyzer will be able to read multiple log files and put them in order automatically.

[bobafind]

http://www.quest.com/funnel_web/analyzer/

This free log analyzer can analyze multiple log files (and put them into chronological order), either in a list or drag and drop; it can also put the entries into a database for historic reporting and has lots of other features. I've just started using it and I like it due to the fact that it's free and feels professional.

The free verision is the full version, just with no tech support. It is freeware, not shareware or trialware etc.

[exam]

Thanks bobafind, I'll be checking it out. Hopefully you can define the log format because my webhost adds a couple pieces of info and the logs are not one of the standard formats. (I've had problems with some analysers not being able to read them)
-d-

[bobafind]

Make sure to let us know of any issues you come across :)

[Dodger]

I use Sawmill ( http://www.sawmill.net ). This is an excellent analyzer, runs about $100 USD though. They do have a free copy if you will do some testing for them (which is not to painful to do) you just use the thing and keep a journal of your activities (problems you encounter, how intuitive it is to use, etc.) after about 30 days the prog is yours for free, plus you help them improve the product.

[exam]

I actually did have a problem with the funnel web analyzer. My webhost logs are a pretty weird format, It's similar to the CLF, but they have added more columns. Here is an example:

23.129.231.142 - - [18/Sep/2004:00:38:44 -0400] "GET / HTTP/1.1" 200 3434 www.mydomain.org "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FunWebProducts-MyWay)" "-"

I'm not even sure what the last "-" is for, I think it is for a proxy server if there is one, that's my guess, but nothing has ever shown up there. Any ideas what logfile analyzer I can use with this format?

[bobafind]

The link I gave was for the FreeWare Version - the full version can do custom log formats - you pick and choose the fields, and it can support many more log formats. I'm not sure if they are still selling the full version. I'll edit this if they are.

[mopacfan]

I use Sawmill ( http://www.sawmill.net ). This is an excellent analyzer, runs about $100 USD though. They do have a free copy if you will do some testing for them (which is not to painful to do) you just use the thing and keep a journal of your activities (problems you encounter, how intuitive it is to use, etc.) after about 30 days the prog is yours for free, plus you help them improve the product.
Yep, I just sent in my documentation of five hours of testing to get the 5 domain license.

[exam]

With Sawmill can you specify a custom log format?

[Dodger]

Yep, I just sent in my documentation of five hours of testing to get the 5 domain license.

Far out! You are the first person I heard of going thru this. I already have my own license and never bothered to go thru it. Let us know if the licenses come thru or not, okay? Would be interested in knowing how easy it was for future reference.

With Sawmill can you specify a custom log format?

Yes you can. It processes a ton of log formats also. Go to this page (http://www.sawmill.net/features.html) then look at the tail end of it where the title says "Processes Almost Any Log File (http://www.sawmill.net/features.html#10)".

Per the feature FAQ:

If you want to analyze a log in a different format, Sawmill also lets you specify a custom log format. If your log is generated by publicly-available software, we'll do this for you-- just email a sample of your log file to sawmill@flowerfire.com, and we'll write you a log format descriptor that you can plug right in to your copy of Sawmill.

We're continually adding new log formats, so the list above will keep growing.

[exam]

Looks like I'm going to have to give it a try...

[Dodger]

Looks like I'm going to have to give it a try...

Kewl. Let me know how you did later.

This software is so configurable, and the options are so extensive, you will discover new things it can do even months from now. It is just like a game of chess -- takes only a day to learn how to play, a lifetime to master. :D