Recently I have noted down that my pages are loading little slower than it was before. When I saw the status, the most idle time is when it was waiting for google analytics. I get a feeling that GA is making the loading process slow. Any one else felt the same?
It is very rare to see google's servers slowing down your site. It all comes down to who's servers are faster - yours or googles. Unless googles are down (which is very, very rare) I can almost guarantee they are faster than yours, so you shouldn't see any impact from a page load.
It is not making your servers load slowly. Your internet connection might be the problem. Google Analytics is only loaded by those who are accessing you websites and it has nothing to do with the servers that are hosting your websites.
We have been doing a lot of Google Analytics testing and have so far to see any significant download speed challenges for any websites. Unless you have extremes amounts of data I think the solution to your problem should be found elsewhere.