On my game cheats site, www.tgdb.net, I have a "print" function, for each game platform. For example, I have /pc/print.php , /psx/print.php I'd like to use robots.txt to prevent bots from accesing these pages. I think this would work but I'm afraid to add it before asking advice: Dissallow : /*/print.php I got this from http://www.imdb.com/robots.txt where they use it to block certain folders, not files. So my question is, how would I enter a line in robots.txt, if possible, to prevent bots from downloading print.php in any subfolder of the main page, even if links have attributes ( example /pc/print.php?cheat=1000) Thanks..
I've just looked at your robots.txt and I see you've not done anything about this yet. I would list the pages individually: User-agent: * Disallow: /pc/print.php Disallow: /psx/print.php etc
Thanks. That would surely work. I've decided to do nothing about it at this moment because Google seems to like those pages and indexed all of them. I was initially afraid that it may think I have duplicate content or something like that. Bandwith is not a problem at this moment .. so I'll let it "flow" for now.. Your answer is much appreciated
The following as far as I am aware Disallow: /pc/print.php Would not block the robot from accessing the page with query string parameters such as /pc/print.php?cheat=1000 Let me know if anyone knows any different
i have 7 error in my Google web master tool and i have requested google to remove those pages because they are not exist any more. but google is saying i want to block these pages from robots.txt file as well. if i exclude the following would work for me. Dissallow : /category/jobs/marketing-jobs/ Dissallow : /category/jobs/human-recourse-jobs/ Dissallow : /seo-ranking-factors/ Dissallow : /category/jobs/sales-jobs/ Dissallow : /category/jobs/admin-jobs/ Dissallow : /category/jobs/finance-jobs/ Dissallow : /category/motivation/ Please advise.
i have just found a good wordpress plugin for 301 redirect. http://wordpress.org/extend/plugins/redirection/ try that i bet you will love it. i have already installed on my sites.