I also believe that the location does not matter, but the page loading speed may have had some impact.
Does google care? Nope! On the other hand, it will rank you for the country where site is hosted. Or one can just simply set country in webmasters tools and it will do the same, no matter if hosting is 2000miles away, aslong response time is "green".
I would say Google does care about where your site is hosted.. if you are targeting the german market and host it in Canada.. A big NO.. If you are targeting the german market, host your website in germany...
Agreed from what I have read lately they do care and so should you really, why would you want a server on the other side of the world from your visitor? Thats going to mean slower load times for them and more likely to leave your site anyway. But yeah location of your server and domain tld is used to workout what area your website is targeting. Dont think its a big issue, more important that you have reliable hosting with no downtime and fast turn around for requests but it does matter all the same. http://www.youtube.com/watch?v=hXt23AXlJJU
I dont think google has any criteria to track the host of the website.. it is uselesss.. but google might have the information of the host but they dont affect it in the search results...
Server location can help with getting ranked in specific country Google results. i.e. if you want to rank well for a site for french people, then better to host your server in france.