Hi I am trying to make a .htaccess file to redirect google/yahoo/msn to certain pages on a site im doing work for. They have a flash site, but want some normal html pages indexed. This is what i think the htaccess should have in it, but it doesnt work! I have put in my IP address as %{REMOTE_ADDR} 82.69.86.69 so i can test to see if its working. Is that done correctly? Can anyone help or suggest what i may have done wrong? Thanks Matt RewriteBase / RewriteEngine on RewriteCond %{HTTP_REFERER} ^.*yahoo.*$ [OR] RewriteCond %{HTTP_REFERER} ^.*google.*$ [OR] RewriteCond %{HTTP_REFERER} ^.*msn.*$ [OR] RewriteCond %{REMOTE_ADDR} 82.69.86.69 RewriteRule ^$ /htmlsite/index.htm [L]
I don't think you got that right. If you use %{HTTP_REFERER} you just redirect people coming from these searchengines result pages and not the bots themselves. You have to use %{HTTP_USER_AGENT} to identify these bots. RewriteCond %{HTTP_USER_AGENT} (msnbot|slurp|googlebot) [NC] Code (markup): And its a very bad idea to make the bots have other content than normal users. Google might penalise you for that. You better make a sitemap of the pages you want to have indexed and submit that to the search engines and write in your robots.txt where they can find the sitemap. And if you want that the bots don't index certain pages you need to disallow it in the robots.txt So its something like that: RewriteBase / RewriteEngine on RewriteCond %{HTTP_USER_AGENT} (msnbot|slurp|googlebot) [NC] RewriteCond %{REMOTE_ADDR} ^82.69.86.69 RewriteRule ^(.*)$ /htmlsite/index.htm [L] Code (markup):
evera's right. Regarding giving search engines a different version of the page, I think it is acceptable in this case. The search engine will act like a user without flash. I'm assuming that, on your page, you have a "view html version" link for users who don't have the flash plugin or don't want to use flash. Few changes to the mod_rewrite code, mainly the OR flag otherwise the condition would never be satisfied: RewriteEngine on RewriteBase / RewriteCond %{HTTP_USER_AGENT} (msnbot|slurp|googlebot) [NC,OR] RewriteCond %{REMOTE_ADDR} ^82\.69\.86\.69$ RewriteRule .* htmlsite/index.htm [L] Code (markup):
I have tested this method and i still see the flash pages load when i try and load the site via my IP address Do i have to make a robots file with any details in? Or maybe just make a robots file that blocks some pages and gives access to the ones i want indexed ? Thanks Matt
A dot in regex is a special character representing any character. Escaping the dot matches a literal dot. Incidentally, using . instead of \. works because a dot is still an element in the set of all characters. In this case, it did not matter in this case as it works on a technicality, but it is good practice to do this right. e.g. ^82.1.9.120 would incorrectly match 82.129.120.*
Cool thx I've learnt something again And mattjack I don't know what you are doing, this should definitly work.
Ok i have it working ! But i have another problem. On the index page i have other links e.g about.htm contact.htm etc When i click these links it still displays the content from the index.htm file. How would i go about adding in these other pages in the htaccess file so that when the links are clicked they go to the right pages? Thanks Matt