I read all around the internet about references to The Cloud (e.g Google Cloud, Amazon S3 Cloud, Server Cloud). What does it mean exactly?
Its an infrastructure of servers where they pool all of the server cpu/diskspace etc and provide it "on demand". For example the Amazon S3 storage cloud is a global network of servers but you never see the underlying hardware, you just upload your data whether its 1mb or 100TB and they do the rest you just pay for what you use.
Depends on who you speak to. Each company has its own definition of what cloud computing/storage is, for some it is just another fancy name for VPS that can be migrated to another server and for some like Amazon etc. it is sort of distributed computing system where all the available physical resources are available in a virtualized environment and can be used to start virtual servers, which aren't dependent on one particular physical server and thus have certain form redundancy and can grow when there is need for more resources. For more details, check out http://en.wikipedia.org/wiki/Cloud_computing
So it is basically mirroring the same content (website files and data in the case of web hosting business) across different servers in different locations and always provide the contents from the location with least load and closest to the requesting person's location?
No, that is the work of a CDN (Content Delivery Network). Cloud mostly is limited to a single geographical location (single DC) and majority of them simply ensure that they provide a bit of redundancy and ability to scale to larger size, when the need arises (though not all do that).
So basically, instead of having a single CPU running at 90% with the risk of running out of resources soon, the company will just rent two more PCs and have them balance the resources to 30% for each, increasing operation costs, but reducing risk and improving performance. From what I read in Wikipedia, I also assumed this scenario for "the cloud", but I am not sure if I got it correctly, it could be valid for big cloud providers like Google or Amazon. Let's say Google has servers in USA and servers in smaller countries like Switzerland. The demand on the US servers (thus the performance and end-user loading speed) increases for some reason (e.g a seasonal or regional trend). The cloud detects this through an algorithm and copies the partial content to the Switzerland server (only the content that's on demand though, otherwise it would be a regular CDN as you said) and direct Europe visitors to the Switzerland server instead of to the USA server, balancing the load between the two servers. In a similar manner, if a Chinese demand increases it may release some of the USA server resources to reduce the load on the China server. Am I correct on this one?