My programmers are using a URL rewrite on my product pages to make a URL go from this (as an example): /productdetail.asp?Product=4999 to this: /keywords-4999.asp There are links in my site pointing to both version of these pages, and I'm worried that both will be indexed (as duplicate content). I'm trying to update all links to the new version of the URL's, but am wondering: What is the best method to prevent both pages from being indexed? If I apply a <meta name="robots" content="noindex"> tag to my "productdetail.asp" page, will that prevent (or unindex) my URL rewrite version as well? I only want the URL rewrite version to index. Any thoughts or suggestions are appreciated.
Maybe setup a 301 permanent redirect in the htaccess file along with the rewrite. That will tell the search engine that the page has permanently moved to the new location. Any incoming links to the old location will be redirected to the new address and with the 301, the search engine should not record the old address.
I don't know why do you need to prevent bot from being crawled your pages. If you are aware about duplicate content then I recommend you set up 301 redirect each page. This could consume much time depends on the numbers of URL. However, it could help your site in a long run.
That rewrite should convert and redirect that dynamic url into static url. If successfully done, your server should give the new static url instead of the dynamic one.