1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Lots of problems with keyword tracker

Discussion in 'Keyword Tracker' started by NiB, Apr 25, 2005.

  1. #1
    Hello,
    SEMrush
    as the list of keywords that I track grows the keyword tracker works worse and worse.

    1. If I press 'Check all' it doesn't check all. It checks like 10 items, then stops and aborts the check. I have to press 'Check all' again so it continues to check the other items. With all the 300 keywords I'm tracking I have to press it 30! times each day.. and wait and wait (as it gets like 1 item done in 30 seconds).

    2. The results don't reflect the SERPs. Every second day half (strangely it's only half of them) of the keywords show -9999 for Google and a few hours later everything is okay again. The same happens with Yahoo from time to time. So I virtually never have complete stats.

    3. Even if the -9999 problem doesn't occur, the results are very unaccurate. Eg. the MSN stats seems to be 'behind' their web results for like 12 hours.

    To me, the source of all the problems seems to be the use of the SE APIs instead of parsing their web pages. Has somebody changed the tracker code (that I have to put on my website) to parse the SE websites instead of using their apis?

    Thanks in advance.

    Regards
     
    NiB, Apr 25, 2005 IP
    SEMrush
  2. SEbasic

    SEbasic Peon

    Messages:
    6,317
    Likes Received:
    318
    Best Answers:
    0
    Trophy Points:
    0
    #2
    This is against the Google TOS...
     
    SEbasic, Apr 25, 2005 IP
  3. T0PS3O

    T0PS3O Feel Good PLC

    Messages:
    13,220
    Likes Received:
    778
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Parsing the site is against the TOS of the SEs and probablt Shawn's as well. You can write your own script if you want to waste bandwidth.

    Mine only gets stuck when my server chokes on the flood of requests from DP. Doesn't happen often. Maybe you have a slow server.
     
    T0PS3O, Apr 25, 2005 IP
  4. NiB

    NiB Peon

    Messages:
    36
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #4
    How do I waste bandwidth when I write my own script?

    The TOS can't tell you how to fetch the website. It's their own fault when they don't prevent me parsing their site with a script as they could _very_ easily do that. (Eg. by requiring javascript to use the search) If you offer a website, you must know that people will go and fetch it with different browsers / ways. That's like publishing a newspaper and requiring that no black people may buy it.

    Besides I think i have a very fast server. Google loves it for being to fast in comparison to others.
     
    NiB, Apr 25, 2005 IP
  5. T0PS3O

    T0PS3O Feel Good PLC

    Messages:
    13,220
    Likes Received:
    778
    Best Answers:
    0
    Trophy Points:
    0
    #5
    By fetching a lot of peripheral stuff you don't need. An API only returns what you want. A webpage returns a lot of filler. Downloading that is a waste of bandwidth if you only want the ranking. It's internet pollution. It's unethical and morally incorrect to do. It's against all net etiquette.
     
    T0PS3O, Apr 25, 2005 IP
  6. Thewormman

    Thewormman Peon

    Messages:
    95
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Hasn't really worked for me for 5 days now, click check all and it does one if you are lucky. It has always been a bit difficult needing quite a few goes to do them all but now I am looking for something else to use that works when needed.

    Don't get me wrong I am very grateful to shawn for providing something free of charge but sometimes it is best to pay for something and if it doesn't work you can at least shout about them putting it right and not waste loads of time mucking about with it.

    JMO of course
     
    Thewormman, Apr 25, 2005 IP
  7. NiB

    NiB Peon

    Messages:
    36
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Oh yes, of course, this one megabyte or so more traffic each day (that I pay for!) will be a very big problem..

    You don't seem to be aware of the trend that mod gzip is turned off at big providers to save cpu power in favor of ip traffic.
     
    NiB, Apr 25, 2005 IP
  8. dave456xx

    dave456xx Guest

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    I have had same problem here in Oz for past week- won't check Google, Yahoo or MSN- I can't understand why it worked perfectly for the first week & now it fails for all 3 SE's??

    Confused & disappointed

    Dave in Oz :confused:
     
    dave456xx, Apr 25, 2005 IP
  9. NiB

    NiB Peon

    Messages:
    36
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I assume the SEs are doing that intentionally as they know the api is mostly used by SEOs. They wouldn't dare the make their websites that fucked up, because they'd lose lots of users. That's why we should parse the websites to get reasonable stats.
     
    NiB, Apr 25, 2005 IP
  10. NiB

    NiB Peon

    Messages:
    36
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Ok, I've looked at my logs. The problem seems to be that each query to google api takes ages. Eg when I manually enter the url the tracker uses, I have to wait like 30 seconds until I get any results.

    Though, when I ping the api server, it's fast:

    ping api.google.com
    PING api.google.com (216.239.37.104): 56 data bytes
    64 bytes from 216.239.37.104: icmp_seq=0 ttl=242 time=95.8 ms
    64 bytes from 216.239.37.104: icmp_seq=1 ttl=242 time=95.8 ms
    64 bytes from 216.239.37.104: icmp_seq=2 ttl=242 time=95.8 ms
    64 bytes from 216.239.37.104: icmp_seq=3 ttl=242 time=95.8 ms
    64 bytes from 216.239.37.104: icmp_seq=4 ttl=242 time=95.9 ms
    64 bytes from 216.239.37.104: icmp_seq=5 ttl=242 time=95.8 ms

    So as I've said before, they intentionally make their server that slow when real api requests are done. That's unbearable.
     
    NiB, Apr 25, 2005 IP
  11. yfs1

    yfs1 User Title Not Found

    Messages:
    13,798
    Likes Received:
    922
    Best Answers:
    0
    Trophy Points:
    0
    #11
    I have just the Google tracking enabled right now and it flew through my keywords.

    Sounds like its just hit and miss.
     
    yfs1, Apr 25, 2005 IP
  12. gabs

    gabs Peon

    Messages:
    135
    Likes Received:
    6
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Takes me 6 goes... every day....

    But it does it all in the end....

    Normally about 12:50 GMT......


    (300+ k/w)
     
    gabs, Apr 28, 2005 IP
  13. dazzlindonna

    dazzlindonna Peon

    Messages:
    553
    Likes Received:
    21
    Best Answers:
    0
    Trophy Points:
    0
    #13
    Was thinking about what could be done as another workaround (besides having two accounts), and came up with the thought of having the ability to Check All Yahoo and MSN. In other words, instead of only being able to Check All (which would check all for G, Y, and M), we would also have a separate link to only check all for Y and M. When G screws up, due to the API being funky, we could at least have a quick, easy way to have Y and M finish the process. Then, if necessary, we could hand check each G phrase one at a time. As it is, I'm sometimes willing to hand check G, but I definitely don't have the time to hand check Y and M as well. Just an idea I thought I'd throw out there.
     
    dazzlindonna, Apr 29, 2005 IP
  14. dave456xx

    dave456xx Guest

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Sounds like a sparkling idea, dazzlindonna:)))
     
    dave456xx, Apr 29, 2005 IP
  15. Epica

    Epica Well-Known Member

    Messages:
    1,007
    Likes Received:
    95
    Best Answers:
    0
    Trophy Points:
    170
    #15
    Just for the record my recent experience with the keyword tracker is quite similar to the original poster.

    Click check all - takes a good long while - no big deal, but then when it says its done...its not - like 1/3 of the links have updated

    I've just been giving it a go 3-4 times until they are all updated. No big deal - and its not like I'm about to do the alternative like build my own tracker (HAH) or manually check all 3 engines for all my keywords - yuck

    Everyone is sooo happy when they discover the free keyword tracker, and then people go all "Customer service needy" when something doesn't go just so-so.

    I'd just like to say - thanks Shawn - despite any temporary and minor glitches - its the best tracker out there
     
    Epica, Apr 29, 2005 IP
  16. dave456xx

    dave456xx Guest

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #16
    Mine just worked perfectly for the first time in 2 weeks :)

    Dunno why- I never changed anything
     
    dave456xx, Apr 30, 2005 IP
  17. MoreSolutions

    MoreSolutions Peon

    Messages:
    30
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #17
    The Google portion doesn't work only when lookup up for a keyword on my site. For all other sites, it works fine. I even tried deleting and trying again.
     
    MoreSolutions, May 2, 2005 IP
  18. dfsweb

    dfsweb Active Member

    Messages:
    1,587
    Likes Received:
    55
    Best Answers:
    0
    Trophy Points:
    88
    #18
    Same problem with my tracker as well. I am tracking around 650 keywords :D but I haven't been able to check over 100 odd for the past couple of weeks. Not sure what's going on as it was working perfectly fine until the 23rd of last month or so.
     
    dfsweb, May 2, 2005 IP
  19. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #19
    I haven't read the whole thread... however, I just checked my kewords tonight for the first time in a while and got a bit of a scare because it looked like my main site had dropped for my main keywords for the first time in 2-3 years.

    I went to www.google.com to query those same keywords and discovered that my site hasn't dropped at all... right where I've been for at least the past year, probably longer.

    Not sure what that Google API is using for data these days... :confused:
     
    minstrel, May 3, 2005 IP
  20. dave456xx

    dave456xx Guest

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #20
    After working perfectly one day- it flaked out again today. If it is the Google API, why does it effect MSN & Yahoo results? If it is our servers why did it work for a few weeks & then go down everywhere?

    Totally confused??/

    OzDave
     
    dave456xx, May 3, 2005 IP