Hi guys, my website is been coded to disallow the search engines to crawl www.auravedic.com/robots.txt i just cannot locate the robots.txt file in my Cpanal directory i searched every folder including the hidden files but just couldn't locate the file is there any other way to find the file or any other way so that google can crawl my website? Please help
Your file contained following texts. User-agent: * Disallow: / I will suggest you to create new file and upload to the directory.
Your robots.txt is located on the root folder of your site like "public_html" if you are using c Panel.
Thanks but will that work? how long it might take to work? is there any instant tool that i can check that google can now crawl the website.
Yes Of course You can Create new file and update it again. User-agent: * Allow: / Disallow: /public_html/_movedcontent Disallow: /public_html/old/images and Create sitemap submit it.
go to the public_html directory and you can see robots.txt there. download it via filezilla and then change it to : User-agent: * Disallow:
The programs. txt can be located on the underlying folder of this site just like "public_html" if you use d Solar panel.