Not only will you lose that PR but it could potentially be penalized additionally. It doesn't look good to Google when your links are changed/removed from another site.
yes definitely PR depends only upon the no. of quality links you hav and if the quality is gone the would also be gone.
Until now on, I still didn't know yet what's the PR function to our site. Based on my observations, some webpages with the high PR didn't have even a good postion in google's SERP.
I am still thinking to creat great content is a better way to get backlinks, instead of submitting sites to directories!