HTTP_REFERER Discussion

Discussion in 'PHP' started by projectshifter, Feb 10, 2008.

  1. #1
    Okay, I've been coding for years, so this is more of just getting feedback from other coders. I've been reading $_SERVER['HTTP_REFERER'] and GETENV('HTTP_REFERER') to get the referer of the person who send the page to the site. Now the problem is that I've been getting complaints that my numbers are off. I parse out the name of the domain (and a subdomain if there is one instead), and that's the numbers people are unhappy with. I'm well aware that there is a certain margin of error, but somedays it is extremely off. As far as I can tell, there isn't much that can be done about it, and nothing in my experience tells me what I'm doing is wrong and that recording $_SERVER['HTTP_REFERER'] is the only way to go about it internally. For the last 24 hours, I also recorded a URL tag in all the hits, so that not only did it give me the id # of the referer, it now had a URL with them. I scanned and sorted the database, and the COUNT() of the hits with the same id # and the hits where I used LIKE '%domain.com%' were marginally off, I think maybe 12 hits on a site that sent 23k according to the system.

    If anyone has some insight I'm open to hearing it, but the way I look at it is that we're reading the referer variable from PHP directly, and the discrepancy must be coming from a lack of the referer variable being passed (probably due to the ever growing and popular movement to block out referers, which is just shitty for us webmasters in the first place). If apache and php aren't getting the $_SERVER['HTTP_REFERER'] variable, I'm not sure what else I can do, since for the last 24 hours they did get recorded, and the numbers were almost dead on, but I'm hoping that maybe there is something in my many years of coding I haven't come across or that I'm just overlooking because I'm getting complaints thrown my direction because the numbers don't line up. I know for a fact that my out hits are correct, it's pretty basic to add +1 out when you click on a plug, I have no doubt in my mind as to the accuracy of that, but if the in hits are showing a lot lower than they should, people get pissed off. I know on a couple of occasions Google Analytics is showing higher in that the system we wrote, which usually shows lower, but the only thing I could think of to compensate is that Google can read form one site with Analytics to the next, whether or not it has a referer tag associated with it, and so in turn it can tell that it definitely came from site X, where as our system can't read it at all because of no referer tags. Any coders with experience, feel free to comment, gracias.
     
    projectshifter, Feb 10, 2008 IP
  2. imvain2

    imvain2 Peon

    Messages:
    218
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #2
    unless you are blocking with the robots.txt or nofollow, i'm sure the outbounds links are wrong. I have experienced this in the past, and my numbers were way off. The stupid spiders were throwing off my hits.

    Are you recording the stats through javascript or just a direct php include?

    Spiders could be throwing off of your inbound stats.
     
    imvain2, Feb 10, 2008 IP
  3. projectshifter

    projectshifter Peon

    Messages:
    394
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I'm pretty sure the spiders aren't the issues at all. The problem is that there are fewer 'in' hits vs what people think they should be (as in I was told one day they sent 25k hits and they actually showd 13k, but all the referers via $_SERVER['HTTP_REFERER'] were logged in. The out isn't really the issue though, it's the in more than anything. And since I said $_SERVER['HTTP_REFERER'], it's obviously PHP as per my previous post, javascript is a piece of shit when it comes to that kind of thing unless it's a 3rd party script.
     
    projectshifter, Feb 10, 2008 IP
  4. imvain2

    imvain2 Peon

    Messages:
    218
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Well I mentioned javascript just in case, I never know how people are implementing their scripts.

    Could it be as simple as they had no referrers? As in direct visits?
     
    imvain2, Feb 10, 2008 IP
  5. projectshifter

    projectshifter Peon

    Messages:
    394
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Lol Javascript is crap , and this is the PHP forum ;) But yeah, my thought is that they had no referers (which does not mean direct visits necessarily, it can also mean a lot of other thins). I'm looking for other possible explanations that could account for it, which is why I posted this.
     
    projectshifter, Feb 11, 2008 IP