Hope you people are doing well Is there any script or tool that can crawl my entire site and put all of my URLs with their respective "Titles" in a MySQL database? I will be really really thankful, if someone can help me out on this. I can even give a free PR5 link for six months, if someone helps me achieve that. Thanks in advance.
I've got this script on my comp (somewhere) will dig it out when I get my hands on it! Oops! Misread your post, thought you wanted the urls or title in a db when a crawler/bot visits your site. Ignore the above, sorry.
http://www.isearchthenet.com/isearch/ will do the trick for you. It's a complete search engine that will spider your website and store everything in a sql database. More scripts available here: http://www.hotscripts.com/search/6971723.html
I was not able to get that isearch to work out (thanks though), but, I searched and searched and finally come across "Sphider" - I found it cool and was able to do what I was looking for works like a charm Heres the link - http://www.cs.ioc.ee/~ando/sphider/