Hey guys, I have a forum for one of my hobbies. www.airgunforum.ca about 170+ members 2669 articles Its a phpbb forum, mod_rewrite, bot manager mod (shows googlebot has visited my site 16 times but only 1 page has been indexed), sitemap that is auto updated. Also, is there any way i can make it go straight to the forum without having to redirect like im doing now? I think that may be causing the problems.
This is a longstanding issue with phpBB boards. The problem stems from the random session id's generated for each visitor. These session id's are appended to your sites URL's, making your boards URL's look completely dynamic to a bot. i.e. every time a bot visits a page, the URL is 'different' because of the new SID. If you look around the mod forum at www.phpbb.com and browse the mods at www.phpbbhacks.com you should be able to find some extensions that try to remedy this by stripping SIDs from links. Some try to do this just for the various searchbots. Still, though, its a finicky process. Ive been hacking around with my board for a few months trying to get it indexed... and out of about 150k articles googles only showing 40k indexed (and way less using the API). Good luck, and update the thread if you find anything promising!
I actually have the SID removal mod installed. On the first day i had it on i noticed google was slowly archiving page by page.... now it visits but doesnt seem to be archiving.
It could be the redirection script on your site. It might not stop google from getting to your page but once it gets there it might not want to crawl the other pages. I just installed the seo mod yesterday so I hope my forum gets crawled soon.