I am doing content audit for some of my blogs and noticed that there are lots of tag and category pages indexed by Google, thus creating duplicate content. I obviously don't have these in sitemap.xml but Google still indexes these. I know it is not a good idea and can affect your SERP, so here is my question: from SEO perspective is it better to include NOFOLLOW, NOINDEX tags and category pages in robots.txt file or to place conditions in header.php e.g. <?php if ( is_category() ) { ?> <meta name="robots" content="noindex,nofollow" /> <?php } ?> I know that some people argue for latter as you can make these pages noindex but follow (i.e. content="noindex,follow") using PHP, but I could never understand why would you want this???