Robot.txt for Google (error)

Discussion in 'robots.txt' started by wartes, Apr 10, 2017.

0
  1. #1
    Hello!
    Can somebody help me ?
    in webmaster google i see errors like
    site.com/mobile/page23/mobile
    i want allow only site.com/mobile/page23/
    how to Disallow double /mobile ?
    (becose the page site.com/mobile/page23/mobile does not exist)
     
    Solved! View solution.
    wartes, Apr 10, 2017 IP
  2. Nick Chernets

    Nick Chernets Member

    Messages:
    55
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    33
    #2
    Just paste
    Disallow:
    and what you need to disallow,
     
    Nick Chernets, Apr 11, 2017 IP
  3. Patricia Ann Lee

    Patricia Ann Lee Active Member

    Messages:
    1,190
    Likes Received:
    73
    Best Answers:
    1
    Trophy Points:
    95
    #3
    Disallow: site.com/mobile/page23/mobile
     
    Patricia Ann Lee, Apr 11, 2017 IP
  4. wartes

    wartes Active Member

    Messages:
    11
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    58
    #4
    wtf people ? i need to Disallow pages like this site.com/mobile/page23/mobile (over 70000 pages)
     
    wartes, Apr 12, 2017 IP
  5. #5
    A better solution will be to create server-rules to redirect non-existing urls to an existing one, or just a general 404-error page, or just send them to the index or a search page. Manually redirecting or disallowing that many links are counterproductive.
     
    PoPSiCLe, Apr 12, 2017 IP
  6. wartes

    wartes Active Member

    Messages:
    11
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    58
    #6
    (I found) If someones will be have the problem
    just add this is robots.txt
    Disallow: */*/mobile
     
    wartes, Apr 12, 2017 IP
  7. Md.Saddam Hossain

    Md.Saddam Hossain Well-Known Member

    Messages:
    84
    Likes Received:
    4
    Best Answers:
    1
    Trophy Points:
    138
    #7
    Hello Wartes,

    The best solution, you can redirect this page (site.com/mobile/page23/mobile) to site.com/mobile/page23/ and reupdate it from search console. If you want to disallow it by robots.txt, then follow the below:

    User-agent: *
    Disallow: /site.com/mobile/page23/mobile/

    The search engine won't index this page again. If your site built on WordPress, there are many easy ways fix it out. Let me know if you have any question. :)
     
    Md.Saddam Hossain, May 7, 2017 IP