Well it is clear from the google note that: "our crawler is separate from the Google.com crawler, and does not affect your Google search ranking." So the persons who are afraid of rankings being affected should feel free in their minds as their ranking will not be affetced in anyway due to adsense as googlebot is a clear different bot from adsense crawler
Does it mean that if I use Adwords or Adsense frequently, it will not affect my actual ranking? For eg I had seen a few friends who advertise heavily on Adwords, such that their adverts are always the top, their SERP improved too to within the first place.
There will be no affect of using adwords or adsense towards your organic rankings. Yes it's known to all but my personal opinion is that it could affect your ranking a little bit as placing adsense code increase your page loading time.
Thats the same effect if you had put a image , but id does not have any effect on your rankings.Especailly google.