Weird Activity on Google API Returns?

Hello. I found this forum from a link in digitalpoint's responses on Google's API newsgroup, and I hope someone can help me work on this problem I'm having.

I'm wanting to use the API to track, day to day, the standings of our sites and our competitors sites in the top 100 standings for our set of keywords. I have the script written in CLI PHP using the nusoap library. It's worked fine, up until recently (tho I don't think the script is at fault).

Up until February 2004, our results mirrored what you would find in a Google search -- that is, in any given range of 10 (11-20, 21-30, etc.), a given .com site would only show up twice within that range. During February 2004, the API started listing different pages on our site 8-9 times in a row (i.e. results 24-31). While I understand this may be the way these pages are stored in Google, by default Google limits the groupings to 2 per viewed page.

Is this a problem with the API, or are the API results the true "raw" standings within Google? And if these are the "true raw" standings, is there anyway to get the API to send me exactly what a customer would see?

-----

Also, as a second question: Is there a bonafide list of datacenters, and a way to get the API to go to a specific datacenter? I ask this, because I saw the thread elsewhere on this forum about the new PageRank/BackLink information, and the exact IP address you gave to get this information. Between that and your keyword tool, it showed us as #1 for the keyword "antenna tuner". However, when I searched for that keyword on google.com from our business DSL line, this AM it returned our site as #1, but this PM it returned us out of the top 100. I verified it was a datacenter problem by checking google.com from different shell accounts as well as the IP address listed in the other thread.

How big of a problem is it to find totally different results like that?
Is there anyway to check which datacenters have which set of data?
And overall, how many of Google's users are seeing the "bad" data? (i.e. data that does not have us at #1 -- from my check, 5 sites were getting us as #1 and only our DSL was showing the competitor at #1).

Thanks in advance for any help,
Josh Hogan

 

First, by turning the filter option off, is that something in the nusoap and/or PHP script I made that I'm doing inadverantly? If so, what's the flag for turning filtering on?

As far as our own tools, yours is FAR more advanced than the hack I made I just found your keyword tool today, and I'm using it to track our bigger ones. One reason for our own is so we can keep a database, day to day, of all top 100 results. This is overkill, but this is also mainly so we can track our competitors, and also Google activity over time. Also, we want to track more than one page and specific pages in our site (i.e., the antenna tuner overall category, and also our specific antenna tuner pages). This may be an option with your tool -- I must admit I've only given it a 15 minute look, but I love what I've seen so far

I'm hoping to check out the site (and this forum) at length tonight or tomorrow in a greater detail.

Thanks again,
Josh

 

I'm an idiot.

Looking in my script, I found the CreateParams function (I hobbled this script together from another script I found online). There is a 'filter' that was set to 'false'. I'm resetting that, and checking the results over the next few days.

Thanks for the advice on where to look Shawn

 

Well, the results this morning were still non-filtered. So I've tried changing the array in PHP, but still with no help. I've tried changing the value of "filter" to a boolean true and the string "true" with no luck. I've looked in the Google API docs, but they don't say what the field is (they do say that the SafeSearch param is boolean).

function CreateParams($query,$startno)
{
  $params = array(
        'key' => 'yourkeyhere',
        'q' => "$query",
        'start' => $startno,
        'maxResults' => 10,
        'filter' => true,
        'restrict' => '',
        'safeSearch' => false,
        'lr' => 'lang_en',
        'ie' => '',
        'oe' => ''
        );
  return $params;
}

Code (markup):

EDIT: I also found the specific SOAP message sent, and the variable appears to be working properly. From the above params code, the following is sent:

<filter xsi:type="xsd:boolean">1</filter>
<restrict xsi:type="xsd:string"></restrict>
<safeSearch xsi:type="xsd:boolean">0</safeSearch>

Code (markup):

EDIT #2: One more quick edit: After going through the SOAP response, I found this:

<documentFiltering xsi:type="xsd:boolean">false</documentFiltering>

Code (markup):

I assume this means the aboev filtering request sent did not take. Any thoughts?

 

Okay -- all I need to do is figure out a way to get nusoap to send "true" instead of "1".

I changed it to string, but it's still a "1" that nusoap is sending to google.

Thanks for the confirmation though

 

Okay, I finally found the problem. NuSOAP (showing ver 0.6.4 in the readme, but the version in the response is 0.6.3) sets all boolean values to 1 and 0. The information is in the followings lines if anyone else reads this and needs it changed:

3009: if ($uqType == 'boolean' && !$value) {
3010:                                        $value = 0;
3011:                                } elseif ($uqType == 'boolean') {
3012:                                        $value = 1;
3013:                                }

Code (markup):

Just change the 0 and 1 to string 'true'/'false' values and all works again, including Google.

Thanks for the help again Shawn -- and hopefully this will fix a problem for someone else down the line.

P.S. Out of curiousity, does NuSOAP does this because a SOAP/WSDL standard is 0/1, not true/false?

 

Okay. Problem is solved, but I found an interesting problem.

I know Google's in the process of dropping new data to it's datacenters, but I found this problem:

Script Results (debug) With Filtering

Script Results (debug) Without Filtering

The sets of results are totally different, namely the #1s (unfiltered it's us, filtered it's someone else). The ONLY change between the two is turning on the filter and safesearch option. Should that cause the results to come from a different datacenter?

Sorry for all the questions, but this has been a thorn in my side w/ Google for awhile, and I'm finally getting somewhere with it

Thanks again,
Josh