Grabbing Search Engine Terms in .Net

I want to create a contact form that grabs what keyword a person found me by...I know HTTP_REFER contains the last URL, but is there a way to grab keyword? I know web logs have this info, so I'm wondering if it is possible programatically.

Just wondering if this is at all possible, or if anybody else has any suggestions?

Rock on
-Tony

 

I think that the keywords are retrieved from the referring url...
e.g if the referring URL is http://www.google.com/search?q=my keywords
you would have to get the keywords from that string.

 

here is a function that should do it:

    Function getKeyWords()
        Dim searchEngines(4) As String
        Dim queryVar(4) As String
        searchEngines(0) = "google"
        searchEngines(1) = "yahoo"
        searchEngines(2) = "altavista"
        searchEngines(3) = "alltheweb"
        searchEngines(4) = "live"
        queryVar(0) = "q="
        queryVar(1) = "p="
        queryVar(2) = "q="
        queryVar(3) = "q="
        queryVar(4) = "q="

        If Request.ServerVariables("HTTP_REFERER") IsNot Nothing Then
            Dim ref As New Uri(Request.ServerVariables("HTTP_REFERER"))
            Dim p1, p2 As Integer
            Dim strRef, keywords As String
            For i = 0 To searchEngines.Count - 1
                If ref.Host.ToString.Contains(searchEngines(i)) Then
                    strRef = ref.AbsoluteUri
                    p1 = InStr(strRef, queryVar(i)) + Len(queryVar(i))
                    If p1 = 0 Then getKeyWords = "" : Exit Function
                    p2 = InStr(p1, strRef, "&")
                    If p2 = 0 Then p2 = Len(strRef)
                    keywords = Mid(strRef, p1, p2 - p1)
                    keywords = keywords.Replace("%20", " ")
                    getKeyWords = keywords
                End If
            Next
        Else
            getKeyWords = ""
        End If
    End Function

Code (markup):

I haven't tested it but it should work, to add more search engines you would just extend the two arrays (searchEngines and queryVar).

 

you will find that most search engines use 'q=' including altavista, alltheweb, live... yahoo just wants to be different.

 

oh sweet! Thanks! Simple concept...crazy I didn't sit and think about the q.

I'll have to convert this to C# though.

Thanks!
-tony

 

yeh shouldnt be too hard... i think there is a web app (or maybe a few) that will do that for you

 

http://www.developerfusion.com/tools/convert/vb-to-csharp/

 

Camjohnson, there is a more elegant ASP.NET object: Request.UrlReferrer

 

yeah it doesn't seem to work for me, whereas the classic way does...

 

tomarto tomato....

 

Yeah, Request.UrlReferrer acts weirdo on me.

 

It works perfect for me:

If Request.UrlReferrer isnot nothing then

textbox1.text = "Hey! You just arrived from " & Request.UrlReferrer.ToString & ". Enjoy your stay and click the damn ads!"

end if

 

hehe...click the damn ads...nice

So what makes UrlReferrer better than HTTP_REFER?

 

Here is a bit more elegant code

Put this code in a shared sub of some global class. From every page you want to monitor hits, in Page_Load add:

xxxx.xxxx.LogHit (Request.Url.ToString  (or "default.aspx"),Request.UserHostAddress, UrlRefferer)

Code (markup):

before the WriteToLog sub, RefSource will hold referrer url (if not SE), or the search engine name if SE, and keyword will hold the keyword.

  Public Shared Sub LogHit(ByVal pagename As String, ByVal ip As String, ByVal refurl As Object)
        If ip.Contains("127.0.0") Then Exit Sub ' Not to monitor in development
        Dim RefSource As String = "", keyword As String = ""
        Dim seps As Char() = {"?", "&"}
        Dim splitref As String()
        If refurl.UrlReferrer IsNot Nothing Then
            If refurl.urlReferrer.ToString.ToLower.Contains("yourdomain.com") Then Exit Sub ' not to monitor if not first hit of the website
            splitref = refurl.UrlReferrer.ToString.Split(seps)
        Else
            Exit Sub
        End If
        Dim sti As String
        For Each sti In splitref
            If sti.StartsWith("q=") Or sti.StartsWith("p=") Then
                If refurl.UrlReferrer.ToString.Contains("www.google") Then           RefSource = "Google"                   
                If refurl.UrlReferrer.ToString.Contains("search.yahoo") Then RefSource = "Yahoo"
                If refurl.UrlReferrer.ToString.Contains("search.live") Then RefSource = "Search"
                If refurl.UrlReferrer.ToString.Contains("msn.com") Then RefSource = "MSN"
                If refurl.UrlReferrer.ToString.Contains("ask.com") Then RefSource = "ASK"
                keyword = sti.Replace("p=", "").Replace("q=", "").Replace("+", " ")
            End If
        Next
        If RefSource = "" Then RefSource = refurl.UrlReferrer.ToString
        
        WriteToLog(RefSource & " " & keyword)

    End Sub

Code (markup):

Can't find anything that makes UrlReferrer fundamentally better, well maybe for future changes in server parameter names (ASP.NET will update correspondigly), and code clarity.

 

or even:

    Sub getkeywords()
        Dim searchEngines(), queryVar(), strSE, strQV, keywords, searchengine As String
        searchengine = "" : keywords = ""

        strSE = "google,yahoo,altavista,ask,alltheweb"
        strQV = "q,p,q,q,q"
        searchEngines = Split(strSE, ",")
        queryVar = Split(strQV, ",")

        If Request.ServerVariables("HTTP_REFERER") IsNot Nothing Then
            Dim refURI As New Uri(Request.ServerVariables("HTTP_REFERER"))
            For i = 0 To searchEngines.Count - 1
                If refURI.Host.Contains(searchEngines(i)) Then
                    keywords = HttpUtility.ParseQueryString(refURI.Query)(queryVar(i))
                    searchengine = searchEngines(i)
                    writeToLog(searchengine, keywords)
                    Exit For
                End If
            Next
        End If
    End Sub

Code (markup):

 

hehe. good one. I will still put google.com and search.yahoo, to prevent domains like Ilikegoogle.net appear as a search engine Ö)