In referrence to spidering frequency, it necessary or even beneficial to use a "Last Modified Header" with PHP pages? // always modified header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT"); PHP:
Content expiration is better controlled using such headers as Expires and Cache-Control. The HTTP spec says that HTTP caches may use Last-Modified for content expiration if no other information is available. In this case, the content may be considered valid for about 10% of the time interval between Last-Modified and the time the request was received by a cache. So, simply put, how Last-Modified is interpreted may be different from a crawler to a crawler. Considering the 10% suggestion, though, I would say that it may hurt more to have Last-Modified than not to if it's not accompanied by one of the expiration headers (i.e. Expires, Cache-Control, etc) because some caches may consider such responses as valid longer than one would expect. On the other hand, if the response doesn't have any expiration headers, caches are not supposed to keep such content longer than 24 hours (they may keep it longer, but have to accompany such content with a warning 113 after 24 hours). J.D.