<meta name="robots" content="all"> this is what i gave in my robots meta tag. And since then my forum has had no googlebot visits. Is this a coincidence or is this a mistake? Which is preferrable? <meta name="robots" content="index, follow"> or the one i specified.
The best thing to do IMO is just to leave that meta tag out of your page entirely if you want all your pages indexed and the links followed as that is the default action of all spiders if they do not find something that tells then not to index or follow.
That may well be the cause of your spidering problems. I agree with Mel. Just get rid of the tag entirely unless you are trying to restrict spidering behavior.
If you use the tag <meta name="robots" content="index,follow"> as cozza stated, then you are completely covered. The statement tells the bot to index and follow links. Why risk anything else, it's the correct meta?
I use <META name="Robots" content="index, follow"> On those pages I want indexed and <META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW,NOARCHIVE"> On those I want left alone. One of the many SEO tools out there puke up errors if you don't have a robots meta.
Bingo - right there! If you want all pages indexed don't even use it.... Why let the robots think you care or know about a robots meta?