Do your blogroll partners have to be related to your nitch or can they be completely different type of blogs? For example if you have a blog on cars and you are running adsense and you have a blogroll parner from the health or vitamin nitch. Would it hurt you for SEO? Would it hurt for adsense relavancy?
First of all, it's niche. It depends on why you'd like to put those links there you want the visitors to click on the links or only for seo purposes. Your link to another site won't pass too much Google juice if it's unrelated to your site's niche.
I would only link to unrelated sites if they were friends, and I would label them as such. It won't do much for you, but a few unrelated links won't necessarily kill you either.