RSS Scrape (w/ permission) to my Site's Search Query... How?

Discussion in 'PHP' started by razz.jazz, Feb 24, 2011.

  1. #1
    So I am scraping content off a site (with permission)

    <?php
    $page = file_get_contents(
        'http://www.google.com/trends/hottrends/atom/hourly');
    preg_match_all('(<a href="(.+)">(.*)</a>)siU', $page, $matches);
     
    // Job’s done!
    // $matches[1] array contains all URLs, and 
    // $matches[2] array contains all anchors
    ?>
    PHP:

    Basically, how can I take the content and put it into my search query? Basically I want to have a little list of it to choose from...

    <div id="latest">   <BR><BR> 
              <span id="examples">Recent Searches: <span id="qph"></span> 
              <a href="http://mywebsite.com/index.php?q=" class="recent_search"></a><a href="http://mywebsite.com/index.php?q=" class="recent_search"></a>          </span> 
            </div><br />
    Code (markup):

     
    razz.jazz, Feb 24, 2011 IP
  2. ka4ok85

    ka4ok85 Peon

    Messages:
    13
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    <div id="latest"><BR><BR>
    <span id="examples">Recent Searches: <span id="qph"></span>
    <?php
    foreach ($matches[1] as $k=>$v){
    echo '<a href="http://mywebsite.com/index.php?q=' . $v . '" class="recent_search">' . $matches[2][$k] . '</a><br/>';
    }
    ?>
    </div>
     
    ka4ok85, Feb 24, 2011 IP
  3. razz.jazz

    razz.jazz Peon

    Messages:
    181
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    hmm.. having trouble getting it to work...

    So I got it to appear correctly... but instead of showing the words, it is showing the google links. so the q= is displaying instead of (for example) shuttle launch, so mywebsite.org/index.php?q=shuttlelaunch
    it is instead showing mywebsite.org/index/php?q=http://www.google.com/trends/hottrends/shuttlelaunch
     
    Last edited: Feb 24, 2011
    razz.jazz, Feb 24, 2011 IP
  4. ka4ok85

    ka4ok85 Peon

    Messages:
    13
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #4
    just change $v to $matches[2][$k] =)
     
    ka4ok85, Feb 25, 2011 IP
  5. razz.jazz

    razz.jazz Peon

    Messages:
    181
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Thanks dude. Prefect :)
    Just one question. How can I make it so the first thing shows up in my search?
     
    razz.jazz, Feb 25, 2011 IP
  6. ka4ok85

    ka4ok85 Peon

    Messages:
    13
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    What do you mean "first"?
    By default if will output is the same order as you make calls...


    If you grab data and output you need use asort() or sort().
    If you store this in DB just use ORDER BY in query.
     
    ka4ok85, Feb 25, 2011 IP
  7. razz.jazz

    razz.jazz Peon

    Messages:
    181
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    I think I messed something up... it doesn't work in my search query. I have it where I can search but it doesn't recognize it. I've been tinkering with several designs so I'm not sure why this does it like this....

    like I have a var params = (shuttle) where I was just shuffling searches.
     
    razz.jazz, Feb 25, 2011 IP
  8. razz.jazz

    razz.jazz Peon

    Messages:
    181
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Have you by any chance returned?
     
    razz.jazz, Mar 4, 2011 IP
  9. ka4ok85

    ka4ok85 Peon

    Messages:
    13
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Do you still need help?

    Just post your code and what exactly do you wanna change.
     
    ka4ok85, Mar 4, 2011 IP
  10. razz.jazz

    razz.jazz Peon

    Messages:
    181
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    The code is really long, so I'll do a snipet. Basically I want the firsts result from the scrape to appear in the search first. So when you go to the site, it is already looking up a search result first.


    <?php
    $page = file_get_contents(
        'http://www.google.com/trends/hottrends/atom/hourly');
    preg_match_all('(<a href="(.+)">(.*)</a>)siU', $page, $matches);
     
    // Job’s done!
    // $matches[1] array contains all URLs, and 
    // $matches[2] array contains all anchors
    ?>	
     
     
            <div class='examples' id="latest"> 
              <div id="latest"><BR><BR> 
    <span id="examples">Recent Searches: <span id="qph"></span> 
    <?php
    foreach ($matches[1] as $k=>$v){
    echo '<a href="http://[website].com/random3.php?q=' . $matches[2][$k] . '" class="recent_search">' . $matches[2][$k] . '</a><br/>';
    }
    ?>
    </div>
              
     
            <div id="results"><div id="sharesearch"></div></div> 
            <div class="waitloading"><img src='spinner.gif'></div> 
            <div id="finished">No more results. Try another search!</div> 
      </div> 
          </div> 
        </div> 
        
        <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script> 
        <script>
        
     
          $(function() {
            var examples = $matches[2]
     
            var params={
              q:  examples[0],  // query str
              gender:  'any',   // male female any
              maxlen:  500      // max message len
            };
            document.location.search.replace(/[?&]([^&=]+)=([^&]+)/g,function(_,key,val) {
              val = decodeURIComponent(val).replace(/\+/g,' ');
              if (val + "" !== "undefined") {
                params[key]=val; 
              }
            });
            
     
    PHP:
     
    razz.jazz, Mar 4, 2011 IP
  11. ka4ok85

    ka4ok85 Peon

    Messages:
    13
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #11
    So do you store this data in DB(or in files)?

    If not you need perform this request to google every time...
     
    ka4ok85, Mar 4, 2011 IP
  12. razz.jazz

    razz.jazz Peon

    Messages:
    181
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Not at all. Don't want it to be stored, or need it.

    I would just like to have the first "scrape" appear right the search.

    Isn't there a way to set q = $match[1] or somehow to make the first result appear?
     
    razz.jazz, Mar 4, 2011 IP
  13. ka4ok85

    ka4ok85 Peon

    Messages:
    13
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #13
    You can make query using onload event. Otherwise you need somehow cache/store results to show something before user's query
     
    ka4ok85, Mar 4, 2011 IP
  14. razz.jazz

    razz.jazz Peon

    Messages:
    181
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Hmm. How could I do that?

    Another thing I wanted to try, is to make a "recent" posts, that showed the 10 recent search queries. I would have to store that probably?
     
    razz.jazz, Mar 5, 2011 IP