Getting scientific about geo-targeting

Discussion in 'General Marketing' started by Drew Bennett, Oct 26, 2010.

  1. #1
    I own lots of sites (over 200) and often times I need to use geo-targeting for one thing or another. I found it really helps conversion rates for some of my affiliate offers and clicks for Adsense when I use geo-targeting to make my content more relevant. There are quite a few geo-targeting scripts out there and I’ve used most of them. During my *ahem* 9 to 5 job I work in research, so I wanted to test all geo-targeting scripts and databases scientifically and find which is best overall. In this post I’ll describe the methods I used, the data I worked with, and the criteria I judged each on. If you just want to view my results then you can skip to the bottom.

    In my own experiences I’ve used geo-targeting a lot, and I know there are 4 major databases out there (in alphabetic order):

    Ip2Location
    IpInfodb
    Lambda GeoIP
    MaxMind


    MaxMind and IpInfodb offer free downloads for the database itself, but no script or real easy way to use it. Therefore, even if you download the free database you will likely end up buying a third party script to actually use it. Also, the free versions are less accurate, so to make this a fair comparison I purchased the paid versions of these databases. For purposes of this comparison I will use the most popular companion script for these two databases.

    For my comparison I judged by three categories: accuracy, functionality, and speed:

    Accuracy: I contacted 20 of my online friends who agreed to help me with this. Basically I wrote a small script that they agreed to run on their computer that tests all 4 databases against their known location (they input the city they live in and it finds the distance between the detected location and the actual location.
    Functionality: I made a list of common use cases I often need, and turned them into a matrix of functionality to compare each database.
    Speed: On high traffic sites, this is a huge issue. I wrote a test script to put my test server under heavy load to compare database lookup performance. In this test, I used the single table database when available.

    Without further ado, here are my results

    Here are the results of my accuracy test. The lower the average distance is the better. A high standard deviation means that it could be way off on a few and pretty close on the rest.

    [​IMG]

    These tests were done with help from people in different cities, countries, and continents. I found that IpInfodb and Ip2location had trouble with locating users in some third world countries whereas Lambda GeoIP and MaxMind performed comparably.

    Here is the full comparison matrix for my functionality test. Please note that these are features I found to be important, features you need may not be accurately represented:

    [​IMG]

    And finally here were the results of my speed test. To simplify the results, I benchmarked each database on the same server under identical load to see the max qps (queries per second) each database could take (higher is better):

    [​IMG]

    I found that using the single table database that MaxMind and Lambda GeoIP were essentially equal when taking margin of error into consideration. For purposes of this test, I will consider it a tie. Non self hosted solutions tended to suffer in this area.

    So in conclusion, here are the results (with price taken into effect)

    [​IMG]

    Here are the links if you want to try them out and report back:

    Ip2Location
    IpInfodb
    Lambda GeoIP
    MaxMind

    I hope this helps someone, being the nerdy type it was rather fun compiling all the data.
     
    Drew Bennett, Oct 26, 2010 IP
  2. JasonTheKid

    JasonTheKid Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Excellent post! Very thorough and insightful.
     
    JasonTheKid, Nov 19, 2010 IP