I have a theory about sites that may disappear from the SEs for periods of time. All sites depend on backlinks, the SEs consider them when determining site credibility, etc. If the sites that point back to you lose relevance or may drop in the SE rankings or disappear then you do not have that backlink-at least temporarily. If many of those websites disappear then you will, too. So if my theory is correct it makes securing good backlinks all the more important just in case sites that link back to you disappear.
Your theory is fact, if sites linking to you loose authority it will have a follow-on effect and reduce your authority. Also if all or most of the links to your site are from a common source, for example all article directories and Google decides to place less "weight" on these types of links it can reduce your authority also. However, this does nothing to explain why a domain (such as one of mine) that has over 100,000 backlinks naturally awarded from thousands of different sources ranging from PR8 would sink from xx,xxx uniques and $450 in revenue per day to just 200 uniques and $1 per day in 10 minutes. The site had over 100,000 rankings, and every one vanished in 10 minutes.. I was actually viewing the live referrer data in the sites back end when it happened, traffic just stopped and clicking on the referral strings from Google of the last 100 visitors showed i was on page 1 or 2 for the various keywords just minutes ago and was now not in the top 1,000. So backlinks being devalued can certainly account from some sites ranking decline, but it wasn't the case in my situation.
I think most agree backlinks can lose some weight and can lower ones position in the serps. But our problems seem more like penalties then just minor drops in serps. When you lose long tailed keywords that compete with less then 20,000 pages, its quite evident. And one would think that such a devaluation of backlinks would be reflected in the exported toolbar pagerank and it doesn't. I've actually gained pr while my serps have not recovered. I hope that their is a problem with Google's algorithm that will be corrected. But one of my sites has been in this same state for about 4 months now. I'm not holding my breath. On another site I had purchased a text link and required it to be nofollowed as Google advises. And that site got knocked out of the serps as well. It's a shame when you follow all the guidelines and still get hit with some sort of penalty.