If your submission is not good, you doing non follow submission, Not high PR submission, Not good content, this factor you have so you can decrease your backlincs
There are many causes of decrease of backlinks. If your forums or social networking accounts are banned or closed then its sure that you have lost all those links. Also links from blog pages which are removed. You should remember the links where you have placed and then compare them all and you will find the reason
I would imagine with the SE's constantly looking to value links correctly, links that used to be valuable have become devalued now. Do you have lots oflinks form directories? If you do these can sometimes either get buried if more and more links are added to them or the SE's stop valuing them. What backlink checker are you using also?