Block a folder except some URL's

Discussion in 'robots.txt' started by soseooooo, Mar 9, 2016.

0
  1. #1
    Hi,

    I have a folder /FR/ with 2607 useless duplicated URL's i need to block with robot.txt and 130 i will redirect in 301.
    Should i enter 2607 URL's line by line or there is a way to write in robot.txt :
    - Block all the folder /FR/ except these 130 URL's

    Thanks !
     
    soseooooo, Mar 9, 2016 IP
  2. MayurKawale

    MayurKawale Well-Known Member

    Messages:
    221
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    170
    #2
    Yes you can Disallow the folder, so all pages from these folder will get Disallow. Here is the syntax.

    User-agent:*
    Disallow: /FR/

    You can crosscheck in Google WebmasterTool in Robots.txt section
     
    MayurKawale, Jun 29, 2016 IP