How does Yandex do its trick? I want to programatically get the contents of a Yandex.com search result. The proglem is that a search page url does not change when you do a search on yandex.com and advance to see more pages. It must be done somehow by javascript. Any ideas?
That would depend on whether the AJAX code is hand-written or if the script uses a library like jQuery. jQuery has an ajax() function. By hand you'd see something like XMLHttpRequest().
They have a list of results but only show a certain number. The show more makes another request through ajax and tells it to fetch the next page of whatever results there were. Here is an example search. http://www.yandex.com/yandsearch?ca...6149533_1353619949953&yu=8569617751353619947&text=test&lr=29646&p=2&ajax=%7B%22b-serp2-list%22%3A%7B%7D%2C%22b-more%22%3A%7B%7D%7D&_=1353620022706 The green is my search text so it says text=what I searched for, test. The red is the page. See the p? This link would bring up page 2 of the results for my search of test. There is a bing azure api search sample somewhere on the msdn samples site. I saw it, perhaps you can search around and find it to see exactly how they coded it. Essentially, what happens is the search is performed and the total result count is provided say 175, but not all displayed. Only a set of 50 (at least on bing) are shown, you then have to calculate how many pages of 50 go into 175 and specify the search to start at record 51, page 2 or provide whats called an offset. An offset of 10 will provide 17 pages of 10 and one page of 5 if we use the mentioned 175 results in the example. Tricky at first but once you work with it's simple.