Hi guyz, how do i remove this url: mydomain.com/viewquestionsanswers.php?START=1&END=10 thorugh robots .txt all the urls in this page is generated like: mydomain.com/viewquestionsanswers.php?START=1&END=10 mydomain.com/viewquestionsanswers.php?START=11&END=20 Is there any way to remove all the dynamically generated urls through Robots .txt from the search engines?. Regards., sarathy.s
If you disallow viewquestionsanswers.php file, it should disallow all the dynamic URLs involving the file.
If you want to keep the main page indexed you could do a user agent detect and supply Google, MSN, and Yahoo each a unique variable, then they would be reassigned that same variable each time they entered the dynamic page so they would never find and index multiple URLs for a single page.
Hello question i know, that it's possible via robots.txt to disallow indexation of all dinamic files, but unfortunately i don't remember command. anybody ,can remind it thank you
I have one directory divided into 2 categories. One shall be indexed, the other not. The URL for the main directory page (where the user chooses from two main categories) is: http://www.mydomain.com/mydirectory.asp The category I do not want to be indexed and cached is: http://www.mydomain.com/mydirectoryy.asp?cat=TheOneNotToBeCached so will: User-Agent: * Disallow: /mydirectoryy.asp?cat=TheOneNotToBeCached work for me? if not, what will?
So for sure would: User-Agent: * Disallow: somepage.php exclude all of these? somepage.php?id=1234 somepage.php?id=12345 somepage.php?id=12345&id2=12345 etc.
I have the same problem with some pages I dont want listed in google, do I use the same method above to accomplish this?
This does not work, it will disallow somepage.php but allow all the variable versions. I'm trying to figure out the same thing. I just found something else I am going to try Disallow somepage.php* the * is suppose to equal any characters of any number. Seems like that should work If I can't get a better way, I'll have to move the file into a folder and disallow the folder. What a pain!!
Looks like this can help you be dynamic mydomain.com/viewquestionsanswers.php?START=1&END=* Should fix your problem If you only want the one page ignored you can state the exact page disallow: mydomain.com/viewquestionsanswers.php?START=1&END=10