BookAdder: add tens of thousands of relevant pages in minutes

(BookAdder is the direct descendant of the package formerly called "Freebie"; the name was changed only because too many email spam filters bounce any message with the word "freebie" in the subject line.)

How search engines rank sites is famously mysterious, but a point that seems universally accepted is that, all other things being equal, the more pages a site has, the better it's likely to do in the SERPs--provided that those pages are reasonably relevant to its theme. And most will also agree that if a site's pages change with reasonable frequency, that helps too.

The BookAdder package will, with only some modest one-time set-up effort, add roughly 45,000 pages to most sites. Those pages will be real, honest-content pages that are at least mostly relevant to your site's theme and many of which change content at least somewhat on a frequent basis--no search-engine spam.

All that is achieved by simply adding to your site an Amazon-based bookshop that specializes in books that are relevant to your site's theme. The package includes tools to help you select and tune a search phrase that will return a sufficient quantity of titles that are relevant. The surprisingly high total of pages is because you are really operating six distinct bookshops--one for each of Amazon's six national divisions--plus listing titles by both ISBN and by title-author, plus including Abebooks used-book listings for those same books (also by both ISBN and author-title).

If you are satisfied with taking the default package appearance--which is, however, even more (and more easily) customizable than ever--you can be up and running within minutes.

The package in this newest version places no files whatever outside its own directories and makes no changes to any other part of your site; all you need do is set it up and point a few links to it. This version also allows multiple sets of bookshops if you have distinct subdomains.

Moreover, the package automagically sitemaps all its pages anew at every update run (which should be done nightly, and can be set up to go unattended), and even auto-submits update notices to Google and Yahoo (and has easy provision to add MSN when they start accepting sitemaps) at every run, so the engines always know about all your pages.

The pages are each distinctive because (among other things) each, besides a display of the book data, includes all reader reviews available. While some titles have few or no reader reviews, most have several and not a few have many, and even Amazon's own per-title pages do not show them all. And every time someone adds another review for a title, that page has nontrivially changed.

The package requires only PHP version 4.1.0 or higher (almost every host has it); it has been tested thoroughly on Linux-based servers (it is currently in use on a dozen or so sites) but, in honesty, not in a Windows-powered server, though it should work as well there unless there are file-access permission problems.

The package is free. It is set up so that any book-sales revenue generated is divided fifty-fifty between you and the package maker. The idea is that if selling books loomed large in your life, you would already be doing it; this package, though a perfectly legitimate bookshop that ought to bring you in some trifling income, is there primarily to provide your site or sites with tens of thousands of relevant pages.

The package is very heavily and thoroughly documented, and you can read the entire set of docfiles on line, to see exactly what the package does and how. The main package page is at: BookAdder The package is supported by email and, within reasonable limits, by telephone.

(This major upgrade shifts entirely to using Amazon's ECS v. 4, which is finally about as reliable as AWS v. 3 was, and adds many helpful tools for setting up and monitoring your new bookshops.)

Incidentally, as of this writing, the package's host site is shown by Google, taking an average over 21 datacenters, as comprising 83,176 pages, of which there are only about a dozen outside of its bookshop.

 

Very nice concept, Owlcroft. The terms & conditions seem reasonable.

One question: My web hosting company (HostPC) flat-out refuses to permit PHP scripts with the fopen command...says it's a wide-open door for bad guys. Unfortunately, quite a few scripts use fopen, which I believe "opens a file." Does BookAdder use fopen?

 

>My web hosting company (HostPC) flat-out refuses to permit PHP scripts with the fopen command...says it's a wide-open door for bad guys. Unfortunately, quite a few scripts use fopen, which I believe "opens a file." Does BookAdder use fopen?

Yes, it does. It is rather difficult to do much of anything in this world without opening files, whether to read or to write.

The assertion that fopen is "a wide-open door for bad guys" seems, shall we say, a bit ingenuous. The chief concern with reading files is that one might read a file on another system (that is, read a file over the internet) and thereby bring malicious code into the operation of one's scripts. That is certainly possible if one is sufficiently careless, but it's scarcely an inherent danger--it's no different from reading user input to a script. You just have to remember to assume nothing and filter everything and mind what you're using the acquired data for.

Frankly, I think you need a better host. (I like Pair Networks, but pick your own. A host that flat-out won't give you remote-fopen capability on request is a host to run, not walk, away from.

You can and should read the package document on security concerns, and, if you like, download the package and review the PHP (that's the nice thing about PHP: it's all out there to be seen). But I think your host is a couple of sheets to the wind.

 

Here is the generic response from my hosting company regarding fopen:

Since HostPC does not allow the use of the PHP file_open command to pull data from offsite, here's a way to do it that works like a charm using CURL.

PHP Code:

<?
/* create instance of curl */
$ch = curl_init();
/* assign the offsite url */
$url = 'http://www.somesite.com/file.html';
/* set some curl options */
curl_setopt($ch, CURLOPT_FAILONERROR, 1); // fail if error
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);// allow redirects
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1); // return data into a variable
curl_setopt($ch, CURLOPT_TIMEOUT, 3); // times out after 4secs
curl_setopt($ch, CURLOPT_URL,$url); // tell it what to get
/* ok, fetch the data and pass it to a holding varible */
$result = curl_exec($ch); // run the whole process
/* close curl */
curl_close($ch);
?>

Now the varible $result contains the data you need and can be processed in anyway you see fit as if it's been called with a file_open() command.

This can easily be turned into a function or a class.

This doesn't mean a lot to me, but maybe it does to others.

 

> Here is the generic response from my hosting company regarding fopen:
>
>Since HostPC does not allow the use of the PHP file_open command to pull data from offsite, here's a way to do it that works like a charm using CURL. [etc. etc.]

That's all well and good, save that CURL is not integral to PHP, and requires that a given host's PHP setup has compiled in the optional libcurl package. While most probably have, my experience with the problems users have with hosts suggests that there are many barely functional hosting operations out there, and I was reluctant to rely on every user having libcurl available. I suppose that in my next update I can test for the presence of the CURL functions and use them if they are available--complications atop complications.

("With that there engine thingy inside your car, you could, if you were a total fool, go dangerously fast on curves, so we've disconnected it--but we've got this really fine mule and hitch you can use to get your car around, an' it's way safer.")

 

It just occurred to me to ask if their prohibition is expressly over the PHP fopen command (there is no file_open command), or is more generally against reading in remote files, as, for example, by way of lower-level work using the fsockopen and fgets commands?

In BookAdder, fopen is used with an http protocol (as opposed to using it for local files) only to reach other local files (to start them running), not files from outside--but a blanket prohibition on using fopen with http will block that, even though no third-party files are involved. But BookAdder does, of course, fetch XML data files (from Amazon) in the course of running, doing so via socket calls, as mentioned above.

The easiest way to see what's what is simply to get the package and begin installation. About the first thing you do after putting the files in place is to run a script aptly named tryme1st.php, which will tell you at once if there would be any problems (including but not limited to remote file opening).

 

Thanks a lot Owlcroft, this software sound great to give it a try

 

Sounds intresting. I am going to try it on a windows server on a spare bookshop domain I have

 

I do not have an answer to that, but they only mentioned the fopen command when I tried to run a script a few months ago. That is when they provided the curl alternative.

Incidentally, this is the Month of PHP Bugs. www.php-security.org/

 

wow this is bizzare but yeah it sounds interesting

 

As it says on the package page or somewhere in the documentation (I forget which), I myself am using this package, in "right out of the box" form, on about a dozen sites of my own, all of which consequently have nice indexed-pages totals; and I have been using it, and its predecessor versions, for a long time.

The key point, again, is that these are perfectly fair, reasonable, legitimate site pages, most pretty much relevant to your site. (I say "most" because any Amazon search on any phrase will always return a small percentage of weird titles whose relevance to the search phrase is known only to Amazon.) You are not in any way generating "search-engine spam".

You can look in at the bookshop on the package host site, or email OR PM me for a list of my other sites .

 

I just installed it but unfortunately it has too much blah, blah in order to set it up.

Anyway I will give it a further customization when I have time to read all that, thank you

 

> . . . it has too much blah, blah in order to set it up..

It may seem that way from the docfiles, because they go into great detail about what everything does, and further possible customizations; but in fact all you need to do to get up and running is to open up the file customize.php in a text editor and follow the simple instructions right in the file. In fact, all you really need to enter in that file to get started is your site's name (item 1) and a search phrase (item 2); everything else can be tweaked later. (But if you have Amazon or Abe IDs, you should also enter those--items 5 and 6.)

The only other thing is to check that your search phrase is finding enough titles, and tweak it if it is not, and that should only take a very few minutes using the interactive bookcount.php script.

All the rest (mostly but not entirely "look and feel" stuff) can be done later, at your leisure. The idea was to make it very, very simple to get going "out of the box", while yet making possible great amounts of personalizing over the base mode. You can specify everything from your site's favicon to a few particular books that will be recommended on your bookshop front page, from the background and text colors to using AdSense and/or the Digitalpoint Co-op Network, and lots more--but none of that is necessary to getting a basic working version up and going: there are default values provided for everything that you can customize.

 

True, I went through the docfiles and after reading the second I got somewhat bored and confused, but now I'm opening the files you told me so

Thanks for the tip

 

Hi, that sounds interesting I'd like to try your package. Please send thanks.

 

Thanks for this piece of code, I'm having problems with this function on my server

 

Very interesting idea, but considering that the script uses Amazon.com content, wouldn't Google have an issue with the duplicate content of any website which uses it?

 

Sorry, I was away for a few days. Here are responses to several posts:

>Hi, that sounds interesting I'd like to try your package. Please send...

The package is available for download from its home page, which is--

http://seo-toys.com/bookadder-seo-package/bookadder-package.shtml

--where you will also find a fuller description and access to the full package documentation (which also comes with the package itself).


>I'm having problems with this function [file_open] on my server . . .

There is no PHP function of that exact name, but the reference more generally is to using various file commands to open and read remote files (those not on the same server) via HTTP. Because so many hosts today do not trust their own clients, more and more are disabling remote-file-open functionality; when hosts disable those functions, making it impossible for any PHP script to directly read a remote web-site page, a workable alternative method is to use the cURL functions. The relevant portion of the online PHP Manual is at--

http://us.php.net/manual/en/ref.curl.php

--and includes a simple example of fetching a web-site page, very similar to the basic code shown in the post above.

But, while the cURL alternative is useful, it assumes that your host has the optional libcurl package installed in its version of PHP. In the next version of BookAdder I will add a test for the presence of cURL capability and let it use that, if it is there, for reading remote files, so as to preclude this issue. (The next version, in which--so far--that test will be the only significant change--should be out by month's end.)


>Very interesting idea, but considering that the script uses Amazon.com content, wouldn't Google have an issue with the duplicate content of any website which uses it?

No, for several reasons; perhaps the biggest is that the pages each include all Amazon customer reviews available, which even Amazon's own pages do not (they include them a few at a time, page by page). There are other differences as well, such as the links to Abebooks for used copies, the price expressed in multiple currencies, and the general layout.

And the bottom line is the fact: for the very site containing BookAdder, today's look at some 21 G datacenters shows site page counts ranging from 77,300 to 87,000. Page counts for some of my other sites using the package are similar. As I want to emphasize, these are not "trick" or "spam" pages: they are real, legitimate content. You can check those counts, or those for any site, by using this URL--

http ://DATACENTER/search?hl=en&q=site:seo-toys.com

--where for DATACENTER you use the numerical IP address of a particular G datacenter (and, for sites other than seo-toys.com, use the appropriate site name). (G datacenter dot addresses are widely available on the web.) You should get a page showing (in the upper right area) something like:

Results 1 - 10 of about 81,200 from seo-toys.com

That, and my other sites, have been using what is now BookAdder (previously known as "Freebie") for a very long time now, so those are not new or temporary results.

In other words, the proof is in the pudding.