Hi. I need some help. I have a page index.php This page have header.php footer.php and are used with include. My questions. header.php and footer.php need to have noindex from robots.txt? If index.php is indexable what to do with footer.php and header.php? And another one question. I have an website with a lot of pages. And I use for each advertisment an separate php page and this is used in main pages with include. Is ok to have advertisments in a php file? This need to be noindex? Regards, Floryn
As a general rule you should put all included files into a subfolder. You should then make this subfolder password protecting, or totally iaccessible by using .htaccess. There's no harm in noindex'ing them in robots.txt too. If you have a lot of files consider a framework like CodeIgniter, that handles this sort of stuff for you.
You can do the noindex thing with robots.txt, or I normally just use a noindex meta tag. You'd only need to put it in the header. I wouldn't worry about noindex unless you've got serious dupe content issues.
Header.php and Footer.php will NOT BE INDEXED OR CRAWLED unless, you have DIRECT LINK on the webpage If you have just included them, relax and enjoy, don't worry about robots, they will fetch the page as whole. PHP is not readable by anyone except the server
Correct, but mis-information. PHP isn't readable by anyone but the server. But, the likelihood is (I'm assuming, given the name of the files) that they contain at least some HTML, which is served to the browser/user. Your comment about the files needing a 'direct link from webpage to be crawled' is also incorrect. I might link to them from one of my sites, then they get crawled too. It's best practice to take included PHP files out of the web root. On some shared hosting this is impossible, so just secure it with .htaccess and use the appropriate robots.txt rules.
Mike... I think I am in love with you. "PHP isn't readable by anyone but the server. But, the likelihood is (I'm assuming, given the name of the files) that they contain at least some HTML, which is served to the browser/user." PHP ISNT READABLE BY ANYONE BUT THE SERVER - Phew! Read this sentence 4 times , if you still do not get it, read it few more times. PHP = PHP Hypertext Preprocessor, a LANGUAGE which is server-side. Server will COMPILE/INTERPRET PHP and give HTML Output, php = I/p to server ONLY! So, if my index.php has include header.php, crawler will read HTML which means, what is INSIDE index.php which comprises of ENTIRE code , all included files too as 1! "Your comment about the files needing a 'direct link from webpage to be crawled' is also incorrect. I might link to them from one of my sites, then they get crawled too." Genius! I am sure in love with you one of my sites - so your site is something else other than a webpage? Do you know, this PAGE Yes, the one on which you are reading this is also called 'webpage' PHP of this webpage is compiled/interpreted by its apache and what you are seeing is its HTML. Phew!
don't link in your pages: header.php and footer.php . they will not be indexed. php include is OK don't worry.
I'm going to put this down to either inexperience in web development, or an inability to read and digest properly. If you read your post, you said the only way those pages would be crawled was if the webpage including them linked to them - as I said, this is mis-information, any link to them, not necessarily from that webpage, would do the same. Secondly, I don't think you quite understand what the question is. header.php and footer.php contain HTML, so if they're visited directly then there is a possibility they'll be indexed - which is what the question was. He isn't asking whether the header.php and footer.php parts of the index.php file that includes them will be crawled - that's pretty obvious. I suggest you read the question, and helpful answers properly, thoroughly and fully understand what's being said before starting slanderous comments that don't even make sense themselves.
When I leave the argument, it simply means, you don't deserve my time since you lack the level of intelligence to be in dispute with me. - Anon
Thanks all for responses. I know is not neccesary to have a link to a php file to be indexed by search engines bots. My header.php or footer.php contain html that can be viewed if the php file is accesed. I want to make something that this php files (footer, header) can't be viewed in browser and can't be indexed.
Wait................ Hang on a second, (as I proper my hair from shock ur post just gave me) can I see ur webpage link pls.
Hi all! First post on these forums, The best thing to do is change the file extension to something like .inc so it cannot execute on it's own and only be require or require_once into the main body page. Also it's wise to put these files in a separate "includes" or such folder and create a blank index.html and place in it. These means that you do not need to rely on search engines to follow the robots.txt file, it means people cannot start executing part page code and there will be no links to those files so search engines cannot find them. Putting a blank index.html file will "hide" the folder so that a file list does not appear if people or spiders manage to find it. Hope this helps!
awood969 gave nice idea... However, Let me tell u mate... index.php has: Google will NOT crawl header.php and footer.php seperately. it just just scan ur index.php and run away.
Easy peasy, the person who has desiged your template has even sectioned it up for you, the way i see it this would be your header.inc This would be your footer.inc; Anything else would be the main page. There will of course be page variations and you can use $_SERVER environment variables to determine pages and then add custom code. This should allow you to split up your pages, kill any repeating code and cut down on the amount of code you need to be looking through at any moment, this is what I have done on all my websites those commercially built and my private one.Apologies for the massive code blocks!!
Google wouldn't crawl these spaces anyway as Google will only crawl the resulting HTML code, which will look the same no matter how you split your PHP up. This can then be furthered with mod_rewrite to take the headache of trying to get Crawlers to understand dynamic URL's. Google would still crawl everything.