Hi there, I am working on a website that shall contain content that is not allowed to be made available to the public at this point in time. When the content is available a lot of sites are going to crop up about this topic. I would like to take advantage of the fact that I have this information well ahead of the majority, and get indexed by google and start working on my ranking for keywords related to the topic now. So I am looking for advice to get indexed by google while providing an error page for any regular visitors to the site. I know that people will still be able to view the google cache for the site, if there is a way round that it is a bonus, but I can probably live without. My main concerns are if google works out that I am doing this and penalizes me for it. It seems like a similar situation to sites that let google index the content but require member registration, so maybe people with experience in that sort of area can help. Regards.
I am looking for the opposite I want the content indexed but people not to be able to see it. I am able to code stuff that will just block anybody without the browser google uses for example, but I am afraid of being penalized by google for this, so am looking for any legit/proven ways that work.
You cannot show something to robots and have it indexed and not to the public. That would get your site penalized and probably deindexed. Search engines catalog content for the public, period. How you want to roll out your site is not their concern.
This is something very similar to cloaking... Just go for ethical methods instead of wasting your efforts in finding out way out of fooling search engines, they are pretty smart and will surely catch your tricks.
I agree with this totally. Any difference between what the crawler sees and what humans see is cloaking, especially if it is keyword rich content.
It is like showing the taxman that my income is taxable and then telling I am not paying the tax. (You cannot have content indexed and not show in google, it does not work that way. The main objective of google is to crawl, index and show to the public its a search engine..)