I want to set "noindex" in my site, but there are so many pages so I used X-robots-tag. I already tried, but "noindex" isn't set.(checked by search console Fetch as google) so please tell me how to rewrite x-robots-tag. I tried 2 patterns as below. ① RewriteCond %{REQUEST_URI} (^/aaa/.+|^/bbb/ccc/.*|^/ddd/[a-z]+|^/bbb/ddd/[a-z]+|^/eee/[a-z]+|^/bbb/eee/.*|^/fff/eee/[0-9]{2}/[0-9]+) RewriteRule .* – [env=NOINDEX_HEADER:1] Header set X-Robots-Tag "noindex" env=NOINDEX_HEADER ② RewriteCond %{REQUEST_URI} ^/aaa/.+ RewriteCond %{REQUEST_URI} ^/bbb/ccc/.* RewriteCond %{REQUEST_URI} ^/ddd/[a-z]+ RewriteCond %{REQUEST_URI} ^/bbb/ddd/[a-z]+ RewriteCond %{REQUEST_URI} ^/eee/[a-z]+ RewriteCond %{REQUEST_URI} ^/bbb/eee/.* RewriteCond %{REQUEST_URI} ^/fff/eee/[0-9]{2}/[0-9]+ RewriteRule .* – [env=NOINDEX_HEADER:1] Header set X-Robots-Tag "noindex" env=NOINDEX_HEADER <caution> ・I want to set noindex which directory biginning from "/aaa/" , except for /aaa/ (TOP). ・directories biginning from /ddd/ OR /eee/ is "noindex" target pages, but its target is which follows alphabet. ex./ddd/abcd → noindex /ddd/1123 → index ・PDF noindex is valid.(checked by search console Fetch as google) code is as below. <Files ~ "\.(swf|fla|pdf)$"> Header set X-Robots-Tag "noindex" </Files>
To inform search engines not to index your page, just add the following to the </head> section: <meta name=”robots” content=”noindex, follow”> The second part of the content tag shows all the links mentioned on this page must be followed, which will be discussed below: Alternatively, the nofollow tag can be used in an X-Robots-Tag in the HTTP header: X-Robots-Tag: noindex