Small enough that you can minimize your competition and large enough that you still have an audience. Long story short... the answer to your question is different with absolutely every niche.
How long is a piece of string? How are you planning to monetize the blog (if at all)? If you're using adsense I'd recomend you do some research to see how many relevant adds come up. If you can't get enough relevant adds then you'll need to widen your niche.
You might be able to find some information at http://www.entrepreneurs-journey.com/933/do-you-really-need-a-unique-niche/