Exactly how does google determine relevancy off incoming links (apart from of course anchor text). How do they determine the topic of the page in where your link is placed? Is it primarily through the pages title tag or the actual content and does the site need to include your target keywords or can google use a thesaurus like algorithm when it comes to theme relevancy - meaning the page doesn't necessarily need exact target words, but just needs to be on topic.
Page title, content, and many other factors would be taken into consideration determining what a page is about.
As webcosmo noted the page title, surrounding text, etc. play into relevancy. But I've checked the backlink profiles of a good many sites, and I can tell you that any follow link counts - even on non related niche sites. This may change in the future, but for now nearly every followed link is counted.