I've seen quick immediate and spectacular drops in rankings, I've never seen a site slowly fade away unless the owner put no work into it - links, content, etc.