Don't panic, just keep on link building and updating your site, remember google is a moving target, nothing says the same, and you keep on bouncing around the whole time. it is normal! keep on working and you will be back soon
If your site is new or has just been restructured, this is not unusual. Don't panic, but just keep working on your site.
if you gained too many links with the same anchor text recently you might receive a small penalty from google but ishould dissapear in time.