I run a site that gets well over 1 million impressions a day, or 60K+ impressions an hour. I was looking into caching mechanisms and decided to simply write the cache to a Ramdisk that was software based and have it served from it. This increased performance quite a bit and the site is lightning fast. Now, I hate relying on software drivers to do work for me as they can crash quite a bit, and don't always support everything you want to do with it. So my questions to you: Would it be a good idea to plug in a 16GB Voyager USB Flash drive and serve the cache (generated html) off of this instead? Would placing 250K thumbnail files and having all the users read off this drive along with the cached html files cause bottlenecks on the drive itself? How would you go about this? I understand that a USB drive has low seek, but I do not know if it can handle thousands of requests and hour and not die on me within months. I'm wishing that the current RamDisk solution I've been using didn't crash my IIS server when using over 4GB. I think it has to do with the NTFS requirement, not sure. The coder is debugging it. Anyway, suggestions as of what could be used for low latency, fast, stable and constantly read from devices would be great. Windows Server 2003 x64 Enterprise Edition Dual Quad Core Xeon 5320 @ 2.33Ghz, 1333FSB 12GB of RAM, Max of 24GB 500GB OS drive, (2) 150GB raptors, (3) 750 Seagates.
We cache the files as HTML because of the mysql load being introduced if we don't. The pages viewed are not different from user to user, making them dynamic would be rather pointless? (Is this what you meant?)
Cache data in memory as some users previously suggested. Memory is inexpensive these days - add more to your server and you will do fine.