The last couple of days I was trying to find a new robot tracker for bot and spiders. I came across this great solution from two guys from France, which basically allows you to do it from Google Analytics. I translated their findings into English on the blog. Remember, a Web Analytics that is based on Javascript cannot track bots, so you will either need to implement this solution or get a bot tracker. Our recommendation for a software focused solely on bots is CrawlTrack, very impressive so far!
You can also use a database driven stat system, (like one with your hosting company) to track these stats.