Designing a pagerank checking tool

Discussion in 'HTML & Website Design' started by Nima, Dec 7, 2006.

  1. #1
    Hey,

    I'm trying to make a tool for submitting to directories.

    It is at its first stage,

    I want to put a small pagerank picture beside each link. How can I do that in my website? (how can I grab that information and post it on my website?)

    If you point me to the right direction, I'll go and study it.

    Thanks
    Nima
     
    Nima, Dec 7, 2006 IP
  2. drig

    drig Peon

    Messages:
    4,188
    Likes Received:
    175
    Best Answers:
    0
    Trophy Points:
    0
  3. Nima

    Nima Well-Known Member

    Messages:
    3,489
    Likes Received:
    243
    Best Answers:
    0
    Trophy Points:
    175
    #3
    But I dont see any pagerank on there.

    I tried to look at the code from some other websites that offer similar tool, but didn't have any luck
     
    Nima, Dec 7, 2006 IP
  4. Nima

    Nima Well-Known Member

    Messages:
    3,489
    Likes Received:
    243
    Best Answers:
    0
    Trophy Points:
    175
    #4
    I figured it out.

    Thanks
     
    Nima, Dec 7, 2006 IP
  5. drig

    drig Peon

    Messages:
    4,188
    Likes Received:
    175
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Mind posting how you did it? Thanks.
     
    drig, Dec 7, 2006 IP
  6. Nima

    Nima Well-Known Member

    Messages:
    3,489
    Likes Received:
    243
    Best Answers:
    0
    Trophy Points:
    175
    #6
    I decided to use phplinkdirectory as the base of tool and edit everything as I go on. It already has the pagerank module.

    My next step is the following:

    - I want to check the listed directories to see whether a certain website is listed or not.

    Any idea how?
     
    Nima, Dec 7, 2006 IP
  7. ednit

    ednit Peon

    Messages:
    152
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #7
    If you use per/cgi there is a module that can do it also:

    WWW::Google::pageRank

    or:

    http://search.cpan.org/~ykar/WWW-Google-PageRank-0.12/lib/WWW/Google/PageRank.pm

    I use it locally to update pagerank for a directory submission service I run, not sure of server resource usage; that's why I use it on my machine instead of my shared hosting account.

    It seems to work well & provides accurate results.
     
    ednit, Dec 7, 2006 IP
  8. Nima

    Nima Well-Known Member

    Messages:
    3,489
    Likes Received:
    243
    Best Answers:
    0
    Trophy Points:
    175
    #8
    ednit,

    can you explain a bit more please?

    or refer somewhere that I can read and learn the process (i dont know anything about it) :(
     
    Nima, Dec 7, 2006 IP
  9. ednit

    ednit Peon

    Messages:
    152
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Yes,

    Forgive the spelling . . not sure what my issue is tonight.

    First off, though: this is not a "plug and play" solution for you, this might help: but you cannot copy & paste code here as it will not work for you as is. . . it's more of an idea.

    If you can run cgi scripts on your website, the above "program" will check & return the PageRank of specific website addresses (URL's). You can either check many url's with it or only one, it depends on how it's configured.

    I will paste my code that I use on my computer, but it would have to be configured to work with your setup. I wrote this for my setup, which is where I have my directory urls stored in a mysql database:

    
    #!"C:\web\xampp\perl\bin\perl.exe"
    
    require "main.pm";
    
    use CGI::Carp qw(fatalsToBrowser);
    use CGI;
    use WWW::Google::PageRank;
    my $pr = WWW::Google::PageRank->new;
    $in = new CGI;
    $do = $in->param('do');
    $id = $in->param('id');
    print "content-type: text/html\n\n";
    
    
    ### UPDATE PAGE RANK OF DIRECTORIES ###
    $before_count = 0;
    &opendb;
    $lo = $dbh->prepare("select id,url,pagerank from directories");
    $lo->execute();
    while ($qw = $lo->fetchrow_hashref()) {
    $Did = $qw->{'id'};
    $Durl = $qw->{'url'};
    $DBpr = $qw->{'pagerank'};
    
    $CurrentPr = $pr->get($Durl);
    
    if ($CurrentPr ne "$DBpr") {
    $before_count++;
    print qq{<hr>Directory URL: $Durl<br>Old Pagerank: $DBpr<br>New Pagerank: $CurrentPr<br>};
    $dbh->do("update directories set pagerank='$CurrentPr' where id='$Did'");
    }
    sleep 2;
    
    }
    &closedb;
    print "<h3 align=center>$before_count Directories Changed!</h3>";
    exit;
    
    Code (markup):
    This, which cannot be used as is for your purposes, will pull the directory URL from the database, check the pagerank, and if the pagerank has changed for a specific directory since I last updated my database, the correct pagerank is updated for the directory url.

    While this may not be exactly what you're looking for, you can easily get "real time" pagerank status by modifying the above code. You do not have to have the info in a mysql database, either: even a flat file (text file) database would work.

    The only issue is whether it would cause issues on your hosting account if you are in a shared hosting environment. You should check with your hosting company before running a script like this, depending on how many URL's you are tracking the pagerank for, because it may use a lot of system resources, I don't know how it will be in your situation.

    A competent perl coder could make something for you with the above very quickly, I'm more of a hack that has learned a bunch of stuff over the years.

    This is only for the pagerank part of it, I'm sure there is something that can be created to crawl a website and look for a specific URL, though I have never known of a software off hand to do this.

    -Rob
     
    ednit, Dec 7, 2006 IP
  10. Nima

    Nima Well-Known Member

    Messages:
    3,489
    Likes Received:
    243
    Best Answers:
    0
    Trophy Points:
    175
    #10
    Rob,

    Thanks a lot. I wanted to run this on my website so that others could use it too. But if it's going to take a lot of resources, I think I'll just run it on my own computer too.

    I'll try to play with your code in PERL and see what i can do with it.

    Thanks again,
    Nima
     
    Nima, Dec 7, 2006 IP
  11. Nima

    Nima Well-Known Member

    Messages:
    3,489
    Likes Received:
    243
    Best Answers:
    0
    Trophy Points:
    175
    #11
    Rob,

    How do you keep track of which directories you've submitted each website to? That is basically what I want to do.


    Thanks
    Nima
     
    Nima, Dec 7, 2006 IP
  12. ednit

    ednit Peon

    Messages:
    152
    Likes Received:
    10
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Nima,

    My setup is a bit complex; it would take a long time to explain, plus I don't want to publically (or even privately) state the way the backend processes of my service work.

    But, the way I do it is I store all my submission results in a database. I'm a coder, so I have the fortunate ability of being able to make my websites/services do whatever I want them to do. . . I'm not sure if you're a coder, though. I have an administration area where I take care of the submission processes, and once I submit to a specific directory, I hit a button and the website and the directory are stored in a database table and marked as submitted.

    Of course you could just have a text document with the info, but after a few hundred submissions it would be a mess to go through manually.

    Not sure if this is a help for you, but hopefully you can take something from this that will work for you.

    Also, I don't know for sure on the resource usage; I just don't take risks like that because I've been with my current hosting company for over 2 years now, and before that I went through hosts every 4 months. . . but if you don't already have a server setup on your personal computer, I use Apache Friends, and it has worked out very well for me.

    -Rob
     
    ednit, Dec 7, 2006 IP
  13. Textlinksguru

    Textlinksguru Active Member

    Messages:
    611
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    58
    #13
    Textlinksguru, Dec 10, 2006 IP
  14. LucasMS

    LucasMS Peon

    Messages:
    616
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Hey

    I have an pagerank script, but, it only works on my local webserver, with PHP 5 and Apache 2. My ip adress starts with 200 (brazil) ... my server is located in US, with ip 66.xx.xx.xxx, php 4 and the stable version of apache ... the same script does not work on there :(

    do anyone know what is happening?
    very thanks!

    EDITED
    I found this code, is working fine, both localhost and on my server

    
    <?php
    error_reporting(E_ALL);
    define('GOOGLE_MAGIC', 0xE6359A60);
    
    //This class should work on most servers
    class PageRank {
    	function xor32($a, $b) {
            return $this->int32($a) ^ $this->int32($b);
        }
        //return least significant 32 bits
        //works by telling unserialize to create an integer even though we provide a double value
        function int32($x) {
            return unserialize("i:$x;");
            //return intval($x); // This line doesn't work on all servers.
        }
        
        function mix($a,$b,$c) {
            $a -= $b; $a -= $c; $a = $this->xor32($a,$this->zeroFill($c,13));
            $b -= $c; $b -= $a; $b = $this->xor32($b,$a<<8);
            $c -= $a; $c -= $b; $c = $this->xor32($c,$this->zeroFill($b,13));
            $a -= $b; $a -= $c; $a = $this->xor32($a,$this->zeroFill($c,12));
            $b -= $c; $b -= $a; $b = $this->xor32($b,$a<<16);
            $c -= $a; $c -= $b; $c = $this->xor32($c,$this->zeroFill($b,5));
            $a -= $b; $a -= $c; $a = $this->xor32($a,$this->zeroFill($c,3));
            $b -= $c; $b -= $a; $b = $this->xor32($b,$a<<10);
            $c -= $a; $c -= $b; $c = $this->xor32($c,$this->zeroFill($b,15));
            return array($a,$b,$c);
        }
    	///
    
        function zeroFill($a, $b){
            $z = hexdec(80000000);
            if ($z & $a){
                $a = ($a>>1);
                $a &= (~$z);
                $a |= 0x40000000;
                $a = ($a>>($b-1));
            }else{
                $a = ($a>>$b);
            }
    		//echo $a."<br />";
            return $a;
        }
        
        function GoogleCH($url, $length=null, $init=GOOGLE_MAGIC) {
            if(is_null($length)) {
                $length = sizeof($url);
            }
    		$b = 0x9E3779B9;
            $a = $b;
            $c = $init;
            $k = 0;
            $len = $length;
            while($len >= 12) {
                $a += ($url[$k+0] +($url[$k+1]<<8) +($url[$k+2]<<16) +($url[$k+3]<<24));
                $b += ($url[$k+4] +($url[$k+5]<<8) +($url[$k+6]<<16) +($url[$k+7]<<24));
                $c += ($url[$k+8] +($url[$k+9]<<8) +($url[$k+10]<<16)+($url[$k+11]<<24));
                $mix = $this->mix($a,$b,$c);
                $a = $mix[0]; $b = $mix[1]; $c = $mix[2];
                $k += 12;
                $len -= 12;
            }
            $c += $length;
            switch($len){
                case 11: $c+=($url[$k+10]<<24);
                case 10: $c+=($url[$k+9]<<16);
                case 9 : $c+=($url[$k+8]<<8);
                /* the first byte of c is reserved for the length */
                case 8 : $b+=($url[$k+7]<<24);
                case 7 : $b+=($url[$k+6]<<16);
                case 6 : $b+=($url[$k+5]<<8);
                case 5 : $b+=($url[$k+4]);
                case 4 : $a+=($url[$k+3]<<24);
                case 3 : $a+=($url[$k+2]<<16);
                case 2 : $a+=($url[$k+1]<<8);
                case 1 : $a+=($url[$k+0]);
            }
            $mix = $this->mix($a,$b,$c);
            /* report the result */
    		//print_r($mix[2])."<br />";
            return $mix[2];
        }
        
        //converts a string into an array of integers containing the numeric value of the char
        function strord($string) {
    	//echo $string."<br />";
            for($i=0;$i<strlen($string);$i++) {
                $result[$i] = ord($string{$i});
            }
    		//print_r($result)."<br />";
            return $result;
            }
    
        //returns -1 if no page rank was found
        function getRank($url){
            $ch = "6".$this->GoogleCH($this->strord("info:" . $url));
            
            $pagerank=-1;
            $fp = fsockopen("www.google.com", 80, $errno, $errstr, 30);
            if (!$fp) {
                echo "$errstr ($errno)<br />\n";
            } else {
    			echo $ch."<br />";
                $out = "GET /search?client=navclient-auto&ch=" . $ch .  "&features=Rank&q=info:" . $url . " HTTP/1.1\r\n" ;
                $out .= "Host: toolbarqueries.google.com\r\n" ;
                $out .= "Connection: Close\r\n\r\n" ;
                fwrite($fp, $out);
                while (!feof($fp)) {
                    $data = fgets($fp, 128);
    				//echo $data."<br />";
                    $pos = strpos($data, "Rank_");
                    if($pos === false){
                    }else{
                    $pagerank = trim(substr($data, $pos + 9));
                    }
                }
                fclose($fp);
            }
            return $pagerank;
        }
    } 
    $pr = new PageRank;
    echo $pr->getRank("http://www.google.com");
    ?>
    Code (markup):
     
    LucasMS, Dec 11, 2006 IP