1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Can I block an agent by referrer?

Discussion in 'Apache' started by hergo, Aug 15, 2004.

  1. #1
    Hi all,

    This may sound weird, but is there any way to block a user agent by a particular referrer?

    For example, can I prevent Google from following an external link into my site without blocking visitors?

    The reason for this is that a URL of a redirect has appeared in Googles results for one of my sites.

    It has my site cached and all that but when clicked on just redirects over to my site with my URL in the address bar.

    Any suggestions are appreciated

    Arron
    SEMrush
     
    hergo, Aug 15, 2004 IP
    SEMrush
  2. Will.Spencer

    Will.Spencer NetBuilder

    Messages:
    14,789
    Likes Received:
    1,040
    Best Answers:
    0
    Trophy Points:
    375
    #2
    You should be able to block certain referrer agents using Apache's MOD Rewrite.

    Umm... are 'ya using Apache? ;)
     
    Will.Spencer, Aug 15, 2004 IP
  3. hergo

    hergo Peon

    Messages:
    85
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yeah im using Apache.

    Is there any chance i could get a snippet of code that would work?

    Thanks
     
    hergo, Aug 15, 2004 IP
  4. Will.Spencer

    Will.Spencer NetBuilder

    Messages:
    14,789
    Likes Received:
    1,040
    Best Answers:
    0
    Trophy Points:
    375
    #4
    I am no MOD Rewrite guru, but this will only take three commands.

    Google for RewriteEngine, RewriteCond and RewriteRule.
     
    Will.Spencer, Aug 15, 2004 IP
  5. Sorvoja

    Sorvoja Don't hide my ads!

    Messages:
    199
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #5
    You can not block Google by the referer since Googlebot dosn't have a refering address.
     
    Sorvoja, Aug 15, 2004 IP
  6. hergo

    hergo Peon

    Messages:
    85
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Yeah your right. I didnt think about that. Come to think of it my logs dont even show a referring address when the link is clicked on, they do for most links but not this one. This also means i have no idea how much traffic this link is generating (if any).

    But if there is no referring address in my log files, can i still block access from this referrer?

    If not, any other ideas on how i could get this link off of the search engines?

    Particularly Google as it seems to be causing a duplicate content issue.
     
    hergo, Aug 16, 2004 IP
  7. Will.Spencer

    Will.Spencer NetBuilder

    Messages:
    14,789
    Likes Received:
    1,040
    Best Answers:
    0
    Trophy Points:
    375
    #7
    Err... it appears that I am getting my terminology all jumbled up.

    You can block by user agent, and Googlebot does have one of those!

    Sorry. <blush>
     
    Will.Spencer, Aug 17, 2004 IP
  8. hergo

    hergo Peon

    Messages:
    85
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #8
    All's good.

    I've got doubts on whether i can do this though.

    It may just be one of those things i'll have to learn to live with.

    The only thing i can think of is blocking access completely from that link. Singling out Google sounds tricky but id be happy to know how to do it though.
     
    hergo, Aug 17, 2004 IP