google has become adept at devaluing crappy sites. so if you are getting links from crappy sites expect your ranking to go down.
So that's the reason! I was freaking out when i lost a few rankings. I guess there's no reason to freak out now.
That's kinda horrible to know.. It might really be a Google dance cos there are fluctuations with the results. I just hope it will be back to normal.