Google/AOL data in searchable format LIVE!

Discussion in 'Websites' started by 1EightT, Aug 7, 2006.

  1. #1
    http://simplifiedsec.com/KeywordDigger.html

    Here it is guys, the full 18 million query 2GB database from AOL/Google in a searchable format. It's somewhat slow as we are indexing the tables at the moment, but we have a query cache in place that should speed things up as we go.
     
    1EightT, Aug 7, 2006 IP
  2. JC007

    JC007 Active Member

    Messages:
    1,066
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    78
    #2
    Thanks for the link!!
     
    JC007, Aug 7, 2006 IP
  3. AnaB

    AnaB Peon

    Messages:
    1,336
    Likes Received:
    36
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Thanks for the link, just playing around with it :) looks realllly cool...(while i wait for the files to download)
     
    AnaB, Aug 7, 2006 IP
  4. 1EightT

    1EightT Guest

    Messages:
    2,646
    Likes Received:
    71
    Best Answers:
    0
    Trophy Points:
    0
    #4
    no problem. Hopefully it helps some people out.
     
    1EightT, Aug 7, 2006 IP
  5. amitpatel_3001

    amitpatel_3001 Results Follow Patience

    Messages:
    14,074
    Likes Received:
    1,178
    Best Answers:
    0
    Trophy Points:
    430
    #5
    Thats a good opiton, but what i found from the results is that this data is all crap,
    Try with the keywords which you usually search and u will know it ;)
     
    amitpatel_3001, Aug 7, 2006 IP
  6. 1EightT

    1EightT Guest

    Messages:
    2,646
    Likes Received:
    71
    Best Answers:
    0
    Trophy Points:
    0
    #6
    it's definitely interesting to see what people search for.
     
    1EightT, Aug 7, 2006 IP
  7. Nintendo

    Nintendo ♬ King of da Wackos ♬

    Messages:
    12,890
    Likes Received:
    1,064
    Best Answers:
    0
    Trophy Points:
    430
    #7
    Is there a way for it to show how many searches the keywords got?
     
    Nintendo, Aug 7, 2006 IP
  8. dirtdog1960theone

    dirtdog1960theone Active Member

    Messages:
    124
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    58
    #8
    and what do Rank & Clicked URL mean? Do you have the raw text files in a link for download :)
     
    dirtdog1960theone, Aug 8, 2006 IP
  9. amitpatel_3001

    amitpatel_3001 Results Follow Patience

    Messages:
    14,074
    Likes Received:
    1,178
    Best Answers:
    0
    Trophy Points:
    430
    #9
    If thats possible i wont call this data crap ;) :D
    Iam missing that info there and seeing only anonID :rolleyes:
     
    amitpatel_3001, Aug 8, 2006 IP
  10. 1EightT

    1EightT Guest

    Messages:
    2,646
    Likes Received:
    71
    Best Answers:
    0
    Trophy Points:
    0
    #10
    We would have to parse the data and do a count of each unique search term. Possible, but it would take some time given the amount of data present.

    Rank and clicked url.

    The clicked url is the url the person clicked in the SERPS after doing their search. The rank is the position of that url in the serps. 0 means the search engine took them straight to the page (as if you put google.com in the search engine and it takes you straight to google). 1 would mean they clicked on the first result, and so on.
     
    1EightT, Aug 8, 2006 IP
  11. 1EightT

    1EightT Guest

    Messages:
    2,646
    Likes Received:
    71
    Best Answers:
    0
    Trophy Points:
    0
    #11
    We are changing the index type to FULLTEXT on the data so search will be quite fast. No idea how long that will take though. I'm guessing a few hours.
     
    1EightT, Aug 8, 2006 IP
  12. cffoodie

    cffoodie Guest

    Messages:
    27
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #12
    I've got a tool that works a little better. This site only returns 50 rows.. and it's limited..

    dontdelete.com

    It does an Overture-like result set.. I think it's a better layout...

    let me know..
     
    cffoodie, Aug 8, 2006 IP
  13. mdvaldosta

    mdvaldosta Peon

    Messages:
    4,079
    Likes Received:
    362
    Best Answers:
    0
    Trophy Points:
    0
    #13
    Nice dude, real nice... although it's not very helpful from what I can tell. It shows what looks to be the last 50 queries with the search term, but it's not sorted in any valuable way. If I search for the term "widget", I'd like to see it return the most frequently searched for terms with "widget" in them, like Wordtracker & Overture.
     
    mdvaldosta, Aug 8, 2006 IP
  14. mdavis278910

    mdavis278910 Peon

    Messages:
    1
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Interesting...alot easier to use...definitley the best one out there!
    dontdelete.com - really has all of the info - not limited and is actually pretty fast considering all of the info thats there
     
    mdavis278910, Aug 8, 2006 IP
  15. 1EightT

    1EightT Guest

    Messages:
    2,646
    Likes Received:
    71
    Best Answers:
    0
    Trophy Points:
    0
    #15
    We were simply in testing last night. Today we are updating and improving the system.

     
    1EightT, Aug 8, 2006 IP
  16. cffoodie

    cffoodie Guest

    Messages:
    27
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #16
    Assuming you're talking about DontDelete.com..

    It actually does. Quick hint: though a "%" sign in the beginning and you'll get everything. For instance "%widget". The results returned are as follows..
    # distinct users searched # times and clicked to # sites for the give keyword that matches "%widget". Then you can click through no search terms and see the sites that were visited AND their average page rank for that term.

    It shows the top 1000 results.. if there are few by nature of the term.. fewer will show. Let me know if you have other questions or suggested enhancemenets.
     
    cffoodie, Aug 8, 2006 IP
  17. 1EightT

    1EightT Guest

    Messages:
    2,646
    Likes Received:
    71
    Best Answers:
    0
    Trophy Points:
    0
    #17
    1EightT, Aug 8, 2006 IP
    evolving likes this.
  18. 1EightT

    1EightT Guest

    Messages:
    2,646
    Likes Received:
    71
    Best Answers:
    0
    Trophy Points:
    0
    #18
    we are sorting the data based on user feedback, so it may appear slow for a moment. Just try back if it is. It's definitely the fastest i've seen of anyones on the net.
     
    1EightT, Aug 8, 2006 IP
  19. jmman

    jmman Peon

    Messages:
    55
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #19
    Not working for me, I'm getting some php error in file NewMySQL on line 90 and warning on line 89.
     
    jmman, Aug 8, 2006 IP
  20. evolving

    evolving Peon

    Messages:
    226
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #20
    there are a php sql error
    did you configure well the config file ???
    are you sure the password is YES ???
     
    evolving, Aug 8, 2006 IP