I just wondered if any of you who have spiderable PHPbb forums have noticed any change in spider activity the last 14 days?? My forums and posts have been spidered and indexed in google for several months now without any problems, but for the last 14 days googlebot has only spidered and indexed the index.php page, none of the forums or new posts appears to have been visited at all. (last ones from 10th August) Googlebot used to re-index the forums (and most of the new posts) at least once every week. I've done no changes to the forum, and it doesn't use sessions for guests/spiders. I've also tried several different spider simulators, and none of them have any problems, so I can't understand why googlebot doesn't like me anymore... Anyone else who has experienced this? You can find my forum here: http://www.military-quotes.com/forum/
Well ... its gonna take a bit of work to get it to be search engine friendly. You can find a step by step guide at PhpBB Hacks
That was not the question, it is search engine friendly (or at least it was). I've installed those mods before (+ several I've made myself). 20,000+ pages indexed in google, but it stopped indexing everything except the index.php page around the 10th August... I just wondered if anyone else had noticed this, or is googlebot just having a little break from my forum now?
A while ago (can't remember how long) googlebot changed it's User Agent from: Googlebot/2.1 (+http://www.googlebot.com/bot.html) to Googlebot/2.1 (+http://www.google.com/bot.html) Maybe your SID hack doesn't recognize the bot correctly anymore...
I applied the hack to mine where guests don't need an SID. It was very easy and is documented in phpBB's forums.
No, and the cache for each of the forums shows that the newest posts in there are from 10-12 August. Example of cahced forum: http://66.102.9.104/search?q=cache:...ary-quotes.com/forum/viewforum.php?f=9+&hl=en Real forum: http://www.military-quotes.com/forum/viewforum.php?f=9 The SID hack I use removes SIDs from all guests, not just the bots, so that should not be the problem.. I've also tried different spider simulators on the posts/forums, and they look ok.. (No "sid" at the end of the links) http://www.submitexpress.com/analyzer/ Edit: tried to install another spider hack as well now (that looks very good by the way) http://www.phpbb.com/phpBB/viewtopic.php?t=200361 We'll see if that one works.
Damn Bots Have you noticed any decrease in activity from the other search engine bots? From time to time I do notice a "lag" in the updating of the cached pages in my forums. I've never really kept track of the length of the delay or tried to isolate the cause, as Google always seems to make up for lost time when I least expect it. Most of the time new posts/pages are indexed and the cache is visible with in a couple days…but like I said every so often Google seems to take a break. I wouldn't really be concerned unless a lot of time, like 4 - 6 weeks, passed without noticeable updates. One thing I've been doing that seems to help with the indexing of posts in my forums is mentioning, and linking to, specific threads within the forum from one of my blogs or from another site I own.
Googlebot does visit and index the index.php page every 2-3 days. I've made a small sitemap (in html) on my site some days ago, linking to each forum and some of the posts, maybe that helps.. I've just "discovered" some posts from the 15th that's cached, maybe googlebot's just having a week or two off now. That bot tool I mentioned is actually pretty cool.. MSN has visited my forum, and spidered 175 pages since my last post here.. Anyone here who knows all of the IP addresses googlebot uses by the way??
Looks like this thread may have brought back googlebot from it's little vacation... Since my last post googlebot have visited the forum 4 times and spidered over 400 pages. MSNbot has also been quite active, over 600 pages spidered. Edit: Make that 5276 pages... Googlebot is REALLY back from it's vacation now... Another Edit: 6500+ pages, and still going strong...
Redleg, does it come with a step by step installation guide? I haven't installed a mod before. Are there no major bugs? Thanks.
Sorry, didn't see your question until now... It's real easy to install, and everything should be explained in the .txt file that comes with the mod. Feel free to PM me if you have any questions about it.