Stop thinking about the number of plugins you have installed, but, instead, to think about the quality of plugins you have installed. The quantity of the plugins won't going to impact your blog performance, you could have tens of hundreds plugins installed and if each of these plugins only has a few simple lines of code and are not loading resources, such as Javascript, jQuery or CSS files, then your blog would be loading as fast as it was before.