If the only reason for the change is that he lost some high-PR backlinks between toolbar exports, that should not show visibly until the next toolbar export. So some other unknown Google intervention has to be the explanation.
you must check your links. try to have links that google index. submit your articles to sites that are indexed by google.
i am not sure but it might be, because of many outgoing links form your site(external links) or you lost some good backlinks
I removed TLA and removed about 20 sites on my blogroll that are no longer updating and no longer linking to me. My site is a fairly big authority site in the Celebrity Gossip field. The amount of links that google shows has not changed. I hope the removal of TLA and the blogroll clean up will help.