I agree with this - Google never says it will penalize if you have high loading time - all its said as it will consider loading time in ranking. You can check this to fine out loading time - Free Website Performance Tool and Web Page Speed Analysis.. You can use these firefox plug-ins also - Page Speed by Google Read more about the same here - Load time: coming soon as a Google ranking factor?
Yesterday i checked the traffic details for one of the website by clicking on the alexa toolbar where it shows rank for websites in my Mozilla browser. It took me to the Alexa website and showed me ranking and other details what it shows. At the bottom of that page it also showed me the site uploading time. For eg - when i tried to see traffic details for DP it showed me "Average Load Time for Digitalpoint.com Slow (4.453 Seconds), 77% of sites are faster." I think it is best to find site loading time through Alexa.
My Friends. If u want to check your website loading time then visit this tools. http://tools.pingdom.com/
I don't think Yahoo's YSlow has been mentioned here, I'm using that with Google Page Speed and a number of tools that can show my the waterfall of requests. Time spent on optimising for speed is time well spent.