Hey guys, I'm aware that it's a smarter option not to get your site indexed until it has at least a few pages with all the links working fine. Well I've been working on a new site and I have made sure not to put my website address anywhere online so that search engine spiders would not find them. The site's pretty unusable at the moment, it has no working links and its still got a long way to go. My only guess is that the search engines found it through a site with a history of domain names bought? But my main worry is if my site is going to suffer long-term from being indexed and no proper links for the next month?
use robots.txt block the whole site even no page there yet. I always do this, and start build links with robots.exe disallow:/ long before site is ready. Once site ready, remove the disallow:/ and google index all pages almost instantly
I agree with DoDo Me. Use robots.txt disallow all during construction. I do not think it will hurt you if you correct the issue now. If you plan to build mulitple sites, it is good to have a domain just for development purposes. That domain never gets indexed.
Your website can be found via many domain alert websites. They have names appeared upon domain registration. You can prevent search engine bots from crawl and index by using robots.txt as DoDo Me suggested.
Another option is to put <meta name="ROBOTS" content="NOINDEX,NOFOLLOW"> on the header part of any page you don't want to get indexed at the moment. Just edit it into "index,follow" if they are ready..
robots.txt is a good way, but if you are a newbie, prepare the basic contents well, that will be much better.
I think the robots.txt file is the better option The noindex meta tag is used to not Indexing, where robots.txt will stop the crawling of search bots: for unfinished sites, the best option is to stop crawling of all SE bots: Also, one needs to put this meta tag in all pages of a site in order to not the indexing:
Its no use preventing google from indexing ur site. The longer the site is indexed the better it is for you and once your site is ready google is surely going to update ur site again so no issues at all. Nevertheless robots.txt is a better option as stated above.