IP Forwarding using Htaccess

Discussion in 'Apache' started by mattjack, Sep 28, 2007.

  1. #1
    Hi

    I am trying to make a .htaccess file to redirect google/yahoo/msn to certain pages on a site im doing work for. They have a flash site, but want some normal html pages indexed. This is what i think the htaccess should have in it, but it doesnt work! I have put in my IP address as %{REMOTE_ADDR} 82.69.86.69 so i can test to see if its working. Is that done correctly? Can anyone help or suggest what i may have done wrong?

    Thanks

    Matt

    RewriteBase /
    RewriteEngine on
    RewriteCond %{HTTP_REFERER} ^.*yahoo.*$ [OR]
    RewriteCond %{HTTP_REFERER} ^.*google.*$ [OR]
    RewriteCond %{HTTP_REFERER} ^.*msn.*$ [OR]
    RewriteCond %{REMOTE_ADDR} 82.69.86.69
    RewriteRule ^$ /htmlsite/index.htm [L]
     
    mattjack, Sep 28, 2007 IP
  2. evera

    evera Peon

    Messages:
    283
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I don't think you got that right.
    If you use %{HTTP_REFERER} you just redirect people coming from these searchengines result pages and not the bots themselves.
    You have to use %{HTTP_USER_AGENT} to identify these bots.

    RewriteCond %{HTTP_USER_AGENT} (msnbot|slurp|googlebot) [NC]
    Code (markup):
    And its a very bad idea to make the bots have other content than normal users. Google might penalise you for that. You better make a sitemap of the pages you want to have indexed and submit that to the search engines and write in your robots.txt where they can find the sitemap. And if you want that the bots don't index certain pages you need to disallow it in the robots.txt

    So its something like that:

    
    RewriteBase /
    RewriteEngine on
    RewriteCond %{HTTP_USER_AGENT} (msnbot|slurp|googlebot) [NC]
    RewriteCond %{REMOTE_ADDR} ^82.69.86.69
    RewriteRule ^(.*)$ /htmlsite/index.htm [L]
    Code (markup):
     
    evera, Sep 28, 2007 IP
  3. krt

    krt Well-Known Member

    Messages:
    829
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    120
    #3
    evera's right. Regarding giving search engines a different version of the page, I think it is acceptable in this case. The search engine will act like a user without flash. I'm assuming that, on your page, you have a "view html version" link for users who don't have the flash plugin or don't want to use flash.

    Few changes to the mod_rewrite code, mainly the OR flag otherwise the condition would never be satisfied:
    RewriteEngine on
    RewriteBase /
    RewriteCond %{HTTP_USER_AGENT} (msnbot|slurp|googlebot) [NC,OR]
    RewriteCond %{REMOTE_ADDR} ^82\.69\.86\.69$
    RewriteRule .* htmlsite/index.htm [L]
    Code (markup):
     
    krt, Sep 28, 2007 IP
  4. evera

    evera Peon

    Messages:
    283
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Oops :eek: forgot the [OR] you are right krt.
    But are those \ in the IP really nessesary? Just wondering..
     
    evera, Sep 28, 2007 IP
  5. mattjack

    mattjack Well-Known Member

    Messages:
    589
    Likes Received:
    17
    Best Answers:
    0
    Trophy Points:
    130
    #5
    Thanks for that info, i shall test them out today and let you know how they go :)
     
    mattjack, Oct 1, 2007 IP
  6. mattjack

    mattjack Well-Known Member

    Messages:
    589
    Likes Received:
    17
    Best Answers:
    0
    Trophy Points:
    130
    #6
    I have tested this method and i still see the flash pages load when i try and load the site via my IP address :confused:

    Do i have to make a robots file with any details in? Or maybe just make a robots file that blocks some pages and gives access to the ones i want indexed ?

    Thanks

    Matt


     
    mattjack, Oct 1, 2007 IP
  7. krt

    krt Well-Known Member

    Messages:
    829
    Likes Received:
    38
    Best Answers:
    0
    Trophy Points:
    120
    #7
    A dot in regex is a special character representing any character. Escaping the dot matches a literal dot.

    Incidentally, using . instead of \. works because a dot is still an element in the set of all characters.

    In this case, it did not matter in this case as it works on a technicality, but it is good practice to do this right.

    e.g. ^82.1.9.120 would incorrectly match 82.129.120.*
     
    krt, Oct 1, 2007 IP
  8. evera

    evera Peon

    Messages:
    283
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Cool thx I've learnt something again :)
    And mattjack I don't know what you are doing, this should definitly work.
     
    evera, Oct 1, 2007 IP
  9. mattjack

    mattjack Well-Known Member

    Messages:
    589
    Likes Received:
    17
    Best Answers:
    0
    Trophy Points:
    130
    #9
    Ok i have it working ! But i have another problem. On the index page i have other links e.g about.htm contact.htm etc

    When i click these links it still displays the content from the index.htm file.

    How would i go about adding in these other pages in the htaccess file so that when the links are clicked they go to the right pages?

    Thanks

    Matt
     
    mattjack, Oct 3, 2007 IP