Hello, I was just wondering what is the best in this case. I have a site with vbulletin forums and google is indexing thousends of profiles and memberlists, if I disallow this with robots.txt I will have less pages indexed by Google, but more quality pages in proportion. I see that DP Forums and others well ranked forums have profiles indexed. What do you think is the best?
if i were you i would let it crawl by spider it would count as a backlinks for you if you place a link back to your index.letting it crawl by spiders will help your site.