Hey guys... I'm building a site history page for each of the sites listed on our web directory list... I thought I had a genious idea of scraping your website's meta (keywords and descriptions) and using that data on your details page so you get double the keywords and meta - pointing to your site... For example: http://webdirectorylist.smashingdir.com/sites/?siteid=1 but as my bot is scraping more and more data, I'm finding out that the majority of web directory hompeages do not have any meta besides a Title. Is this something that PHPLD and ESyndiCat do by default you think?... or can you think of another good solution to get good meta data per web directory? Thanks guys!
Anyone got any good ideas? or is it just standard not have meta (description and keywords) on your homepage if you run a web directory? Thanks...
There is a place where you can set sitewide meta keyword. By defualt, it should be empty. So people may prefer not to set anythingthere. I dont know the reason if there is any.