Should I have my sitemap set to be crawled daily, or weekly? I figured daily would be best..... What are the pros & cons? I heard daily can slow your site...... What do you guys think???
I don't think you can actually control and tell the bots when to crawl your site. You can however, adjust the frequency using an option available in webmasters tools and set the crawl rate higher. But beware because if you do that, you have to ensure that it does not eat up your bandwidth or anything. Of course higher crawl rate is advantages so that your new page submission is indexed quickly especially for those who update their site often.
yes you can, in webmaster tools as well as in a meta tag... if you post new content daily, have them scan every day if you post once a week, have them scan once a week
Yes daily is best if you frequently update your content. If your website doesn't change much, it would be fine to set it as weekly.
player it doesnt really all depend on that.. i would just let it flow naturally, yet if your site is not 2.0, i would just set it to like every 1 week or so.
Are you sure? I thought there are only options to adjust the crawl rate to be faster, slower or maintain at normal. I don't see how you can select daily, weekly or anything. Maybe I miss out the meta tag part. How do you actually go about doing this??
No, you can't. Quote - "Occasionally webmasters needlessly include "revisit-after" to encourage a search engine's crawl schedule, however this meta tag is largely ignored. If you want to give search engines information about changes in your pages, use and submit an XML sitemap. " http://googlewebmastercentral.blogspot.com/2007/12/answering-more-popular-picks-meta-tags.html
This answer was spot on. However, you don't really have much control over this. Google will tend to come back regular if you're updating regular.
I set it to "Daily"...but, it doesn't slowed any of my sites.. If you often update your site, choose "daily".
Its just a rumour that the daily crawl makes your site slower. Daily crawl is a must is a busy site like forums. If you have a blog and update in once or twice in a week, just set the crawling rate to weekly in Google Webmaster Tools.
You can't set that. Google decide how often crawl your site based on how frequently it found changes.
Google will crawl your site depending on the amount of content you add in your site. If you do not add much content, then slower speed is ok, not if you add content on regular bases, then crawl speed should be faster.
I thought that it worked as a deterent, eg, if you say daily and it has already scanned your site that day then it wont come back till tomorrow.
i think the OP is talking about sitemap generation not robot crawling. the default for my automated sitemap generator which occurs on the server is daily. i personally set it to hourly. my site has 300 articles but it's generated within seconds. i suppose if you're on shared hosting and if you had a site has several thousands of articles maybe daily sometime late night would be better. but for a site in the hundreds, hourly is fine.