Hi, I have a recently had an idea which involved crawling a site and cache part of its content. (sorry to be vague but i am attempting to keep this project under wraps). What are the things i will have to consider while doing this? As i will be storing another sites content i feel there will be many things. Permission from the site owners will be gained prior to crawling the page however is there anything else i must consider?
Read up on DMCA Title II and submit yourself as a designated agent for contact for all alleged infringements.
Google crawls my site every day... If you're only doing a specific site, then there may very well be some issues involved. But if you are doing an archival caching thing, like any other search engine or archive.org, I don't see any problems arising.