We are running many websites in Yii2 application. Currently we are using File Cache for Data Caching. Now we want to use load balancer and put our caching on dedicated server. We will have to store approx 100GB of cached data. What we have tried? File Cache: Working great currently. But We can not move it to seperate dedicated server. Redis Cache: We created a dedicated server for Redis. Since we are on low budget therefore instead of using 100GB RAM server we used 2Core CPU and 4GB RAM server and created Swap space of 110GB. for storing cached data. It worked great but after sometime there was a sudden spike in CPU usage to 100% and Redis crashed. This was started happening daily. I was unable to find the exact reason. Now I have two more options, using MongoDB as Cache server or Separate MySql Server for storing cached data. My question is to experienced developers is that if I use MongoDB or MySql as Cache Server will I get at least File Cache like performance? Currently I am ok with File Cache Performance, I only want to move my cache to separate server.
Your sudden spike in CPU was the swap. You should never swap in memory DB. MongoDB's cache is also in memory, and mongoDB is way worse than redis when in swaps, separate mysql is worse still. Redis is good, just not swap it and do not choose to store it's data on disk. As for 4 GB ram, check out contabo.com they have cheap options for 8-16 gb ram, but that means you need to move everything over to them... they are very much ok for the price. File cache is also ok, i would move the mysql to another server, and leave php and the file cache on the old one...
Redis: The Purebred Caching Beast Redis was literally born for this job. It keeps data directly in RAM, which is why it’s the undisputed king of caching. Pros: Blazing Fast: Since it’s an in-memory data store, read/write latencies are sub-millisecond. It’s as fast as it gets. Native Caching Features: It has built-in TTL (Time-to-Live). You tell it when a key should expire, and it gracefully nukes it. It also handles cache eviction (like LRU/LFU) out of the box when you run out of memory. Rich Data Structures: It doesn't just do simple key-value pairs; it natively supports lists, sets, hashes, and sorted sets. Cons: Expensive Real Estate: RAM isn't cheap. If your cache data footprint is massive, scaling Redis vertically will burn a hole in your pocket. Persistence is an Afterthought: While it does support saving data to disk, if a node crashes hard, you might lose the most recent unsynced data. Kafka: The Heavyweight Data Pipeline Let's get one thing straight: Kafka is an event streaming platform, not a traditional cache. Trying to use it as a direct key-value cache is like using a bazooka to swat a fly. Pros: Bulletproof Durability: It writes everything to disk and replicates it. The best part? You can replay events from the past. This is absolutely brilliant for "cache warming" (rebuilding your Redis cache if it completely crashes). Massive Scalability: It can chew through terabytes of streaming data without breaking a sweat. Cons: Terrible for Point Lookups: Kafka is built for sequential reads. Asking it to "fetch the user with ID 5" (random lookup) goes entirely against its design and is painfully slow compared to Redis. Overkill Complexity: Setting up, maintaining, and tweaking a Zookeeper/KRaft and Kafka cluster just to cache some data is an unnecessary administrative nightmare. The Verdict: If you need a cache, use Redis. No question. But if you want to capture database changes in real-time and push those updates to your cache, use Kafka as the pipeline to feed Redis