I have a vbulletin forum with vbseo installed. I have a site map installed and I use to submit thread/posts/members in this site map to google. about 3 months ago I changed my site map to include only my showthread pages, removing the post and member pages. My site map now includes 75k different pages. when I go to google.com and do site:MYSITE.com it only shows 20k pages in google and it has done for over 3 months now. My question is, why isnt google indexing all 75k of my pages?? Is there something im doing wrong??
A sitemap doesn't mean Google has to index it (hell, we've never even bothered to use a sitemap here). Do you have a lot of external links to your forum (and specifically deep links... not just to home page)?
yeah, i have 250k incoming links, 75k going to my homepage. I took a look in googles webmaster and there are no crawl errors or any other problems so im confused
Google doesn't index entire forums. Have you ever tried getting backlinks from forum profiles? On popular forums, you're likely to get a link from each post you make. But go to a lesser known forum and each post is almost hit or miss. I belong to a car forum that gets a ton of traffic, tons of links, dofollow, etc. I only have 1 link showing up from there and I have 5,000 posts on that forum. Not all of it gets indexed.
Did you check your server logs to make sure it's not spidering? Just because Google spiders a page, doesn't mean it's going to be in their search results though.