I wrote this in the file: User-agent:* Disallow: /cache/ Disallow: /api/ Disallow: /index.php? And i am sure i put it in the root directory. but google still indexed my pages including /index.php? in the url. Is it wrong with the CHMOD of the file? I set it to 777, how to set it? Thanks
When did you update your robots.txt? If you disallow crawling of a URL, this does not automatically remove the URL from the search results, but stop crawling the restricted url. Google takes some time to remove the disallowed URL from the search results. If you really wish to remove that URL from the search results, you can remove it through site removal tool in webmasters tool.