1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

How to check robots.txt on site?

Discussion in 'robots.txt' started by seoworld1234, Apr 21, 2008.

  1. jelaine

    jelaine Peon

    Messages:
    51
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #21
    You can also check in Woorank or Chrome SEO to check a specific website's robots file.
     
    jelaine, Jun 7, 2011 IP
  2. maverick.jones

    maverick.jones Greenhorn

    Messages:
    62
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #22
    maverick.jones, Jun 10, 2011 IP
  3. osCreative

    osCreative Guest

    Messages:
    6
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #23
    Just type domain name/robots.txt and enter file will be shown
     
    osCreative, Jun 18, 2011 IP
  4. upendraets

    upendraets Peon

    Messages:
    106
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #24
    upendraets, Jun 25, 2011 IP
  5. virender.ets

    virender.ets Peon

    Messages:
    41
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #25
    virender.ets, Aug 16, 2011 IP
  6. Shawn Michaels

    Shawn Michaels Member

    Messages:
    54
    Likes Received:
    0
    Best Answers:
    8
    Trophy Points:
    46
    #26
    Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic.
     
    Shawn Michaels, May 10, 2013 IP
  7. Bharat Bhushan

    Bharat Bhushan Member

    Messages:
    15
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    31
    #27
    By webmastertool you can check your robot.txt file....:)
     
    Bharat Bhushan, May 16, 2013 IP
  8. BobbyMcNiel

    BobbyMcNiel Greenhorn

    Messages:
    30
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    6
    #28
    Thanks for sharing this valuable post,it such a very informative.
     
    BobbyMcNiel, Jun 3, 2013 IP
  9. Whitecap SEO

    Whitecap SEO Active Member

    Messages:
    83
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    80
    #29
    You can find or create robot.txt at webmaster tool also.
     
    Whitecap SEO, Jul 23, 2013 IP
  10. Stephen Thomas

    Stephen Thomas Member

    Messages:
    37
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    43
    #30
    Its easy, you can check by writing robots.txt after your domain name. For Example- http://www.domainname.com/robots.txt
    Also, you can check on google webmaster tool > select your website > Crawl > robots.txt Tester.

    Hope this will help you.
     
    Stephen Thomas, May 19, 2017 IP
  11. JAYARAJ

    JAYARAJ Active Member

    Messages:
    1
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    56
    #31
    Thanks i checked as per this post http://www.jaibiz2day.com
    result
    User-agent: *
    Disallow: /feeds
    Allow: /_/rsrc/
    Disallow: /_/
    Sitemap: http://www.jaibiz2day.com:80/system/feeds/sitemap

    travel trends
     
    JAYARAJ, Apr 20, 2019 IP
  12. imranbashir.seo

    imranbashir.seo Active Member

    Messages:
    22
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    86
    #32
    you can simple type "example.com/robots.txt" if it's added it will appear or it will redirect homepage if server didn't found it's very easy to add there but it's a important file you can give the instruction to google that which pages or modules you don't want crawl on search engine.
     
    imranbashir.seo, Jul 25, 2022 IP
  13. Joseph Mack

    Joseph Mack Peon

    Messages:
    16
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    1
    #33
    You can check the robots.txt by using "websitedomain/robots.txt"
     
    Joseph Mack, Mar 7, 2023 IP
  14. imranbashir.seo

    imranbashir.seo Active Member

    Messages:
    22
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    86
    #34
    just type the domain name yourdomain.com/robots.txt you will see if file is added if not you can add in root directory of your domain
     
    imranbashir.seo, Mar 7, 2023 IP