Hello, Can anybody help me with a robots.txt to disallow crawling of URL's that contain a specific parameter? Will this work? User-Agent: * Disallow: /*affid=* I really don't know the appriate syntax for constructing this type of exclusion so any help would be appreciated. In this example I would like the following URL's to be excluded. http://domain.com/abc/index.php?affid=3243 http://domain.com/abc?index.php?color=blue&affid=3433 Please help!
All you need to do it put a nofollow tag in your affiliate urls. e.g. <a href="http://domain.com/abc/index.php?affid=3243" rel="nofollow">link</a>
Then you should first learn about affiliate links and how they are used? Affiliate links are the links that are used by affiliate partners through different mediums. How would you add no follow attribute with each link?
Like I said before, you do it like this: <a href="http://domain.com/abc/index.php?affid=3243" rel="nofollow">link</a> I do it myself on my own sites.
rel="nofollow" attribute is used to instruct cralwers to not follow a particular link from the particular page where the link is placed. But it doesn't mean that crawlers will not crawl and index this link as this link would be used by several other affiliate partners and while promoting the affiliate programs.
Yes, but it doesn't matter to the person who has the affiliate link whether the page is indexed or not because they don't have control over it. More likely than not it will be indexed being a big business with an affiliate program. What they do want to do it stop giving that page any PR juice from their site and they can do that using the rel=nofollow tag.
Did you check the main query for what you had replied? He didn't ask for what you have suggested him.
No, but I am suggesting the way that I know works. If using robots.txt works, but I know that using nofollow works.
A thread is meant to discuss the issues of thread opener and it is quite surprising that you are posting in the thread without even knowing the question. I am not doubting your skills, but a reply should be given in a way that it could help others, not in a way that it can lead others to a wrong path.
I'm not sure why you are being negative. I'm merely giving an alternative solution to a problem. He doesn't have to use it, but he hasn't replied to the thread so we don't know if the one he was using before works.
I am not negative dear. I am just trying to explain you that the solution you had provided would not work in affiliate links scenario. If you have any doubt, do let me know.
As per definition "By adding rel="nofollow" to a hyperlink, a page indicates that the destination of that hyperlink SHOULD NOT be afforded any additional weight or ranking by user agents which perform link analysis upon web pages (e.g. search engines). Suppose you have a website xyz.com and you have a login page xyz.com/login.html. Now you start you affiliate program stating that you would offer $5 per login and people shows their interest to participate in your affiliate program. Now how would you track how many registrations have been done through your affiliate partners. To track this you have to give them a dedicated affiliate URLs like xyz.com/login.html?affid=1254. In Google eyes this page would be a different page, but practically this page would be carrying the content of login.html page, so to avoid duplicate page issues, you would prefer to block all the affiliate urls so that those page won't index and you will use following code: User-agent: * Disallow: /login.html?affid=* Now tell me how would you block these affiliate urls through nofollow attribute? If you have still any doubt, you can invite me on gmail at manish.webmater for a chat session.
Did I say in my comments that I am an SEO Analyst, but I hope after this long discussion you would have come to know about this. Same way, to solve the root of the problem, you have to first study the problem, that you didn't do in this case.