Social Buttons

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 1 of 18
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. Jaden Alvin

    Webmaster Blocked URL's Understanding Issue

    Jaden Alvin, Jul 9, 2014
    Replies:
    12
    Views:
    1,870
    cathandrew
    Oct 18, 2014 at 4:26 AM
  2. aleale

    Is my robots.txt working fine?

    aleale, Sep 10, 2014
    Replies:
    2
    Views:
    666
    maddenitrous
    Oct 3, 2014
  3. gvinaykumar

    How to block a specific folder in robots.txt file?

    gvinaykumar, Jul 17, 2014
    Replies:
    0
    Views:
    755
    gvinaykumar
    Jul 17, 2014
  4. linkladdy

    Error - URLs restricted by robots.txt errors How Fix IT

    linkladdy, Jul 2, 2014
    Replies:
    1
    Views:
    782
    evildon
    Jul 2, 2014
  5. David Lowe

    Can I add specific urls to robots.txt to stop google indexing them?

    David Lowe, Jun 25, 2013
    Replies:
    2
    Views:
    1,757
    BabyDoll123
    Jun 19, 2014
  6. cathandrew

    How to clear these errors in robots.txt?

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    837
    BabyDoll123
    Jun 19, 2014
  7. mjfnd

    how to dissalow an image?

    mjfnd, Apr 29, 2014
    Replies:
    1
    Views:
    794
    BabyDoll123
    Jun 19, 2014
  8. BabyDoll123

    Magento Robots.txt file

    BabyDoll123, Jun 12, 2014
    Replies:
    3
    Views:
    872
    Nigel Lew
    Jun 14, 2014
  9. imaginexhacked

    Blocking auto generated pages with robots.txt

    imaginexhacked, Jun 9, 2014
    Replies:
    1
    Views:
    754
    O-D-T
    Jun 9, 2014
  10. myfastrupee

    Robot.txt file not found on my blog

    myfastrupee, Jun 21, 2013
    Replies:
    2
    Views:
    1,111
    Constructor Decasa
    Jun 4, 2014
  11. jiangnancun

    How can i find those website in which robots.txt disallow google spider?

    jiangnancun, May 13, 2014
    Replies:
    2
    Views:
    796
    neroux
    May 13, 2014
  12. James101

    What is robot.txt file?

    James101, Apr 1, 2014
    Replies:
    4
    Views:
    1,060
    John Dave
    Apr 2, 2014
  13. garry420

    I have at-last found site that creates xml file of https sites

    garry420, Mar 8, 2014
    Replies:
    0
    Views:
    634
    garry420
    Mar 8, 2014
  14. bright776

    how to disallow a folder from being indexed

    bright776, Feb 1, 2014
    Replies:
    3
    Views:
    767
    Ehteshamshaikh
    Mar 7, 2014
  15. cathandrew

    Is it necessary to specify sitemap.xml in robots.txt file

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    685
    Ehteshamshaikh
    Feb 27, 2014
  16. manish_khanna

    Things you should be doing to block bad BOTS.

    manish_khanna, Feb 13, 2014
    Replies:
    0
    Views:
    743
    manish_khanna
    Feb 13, 2014
  17. Angelabailey

    Robots.txt

    Angelabailey, Nov 15, 2013
    Replies:
    14
    Views:
    1,395
    greenoro0
    Feb 3, 2014
  18. SebastienD

    Dissalow folder with the same name as allowed folder

    SebastienD, Jan 15, 2014
    Replies:
    2
    Views:
    1,019
    Nette
    Feb 2, 2014
  19. vatu

    Robots.txt File Help

    vatu, Jan 15, 2014
    Replies:
    8
    Views:
    909
    alsgroup
    Jan 23, 2014
  20. Sunil Keshari

    Robots.txt code

    Sunil Keshari, Jan 9, 2014
    Replies:
    3
    Views:
    1,152
    ashokdhan
    Jan 14, 2014
  21. Vel

    I need help on how to Noindex categories, tags, author, archive

    Vel, Jul 25, 2013
    Replies:
    6
    Views:
    1,233
    tymsh
    Dec 9, 2013
  22. chadhaajay

    Please review my robots.txt code

    chadhaajay, Oct 10, 2013
    Replies:
    2
    Views:
    858
    makeit easy
    Nov 17, 2013
  23. pradip200

    Robots.txt for SMF Forum

    pradip200, Mar 23, 2013
    Replies:
    3
    Views:
    3,092
    donjajo
    Nov 9, 2013
  24. rob587

    Would a robot.txt fix this SEO issue?

    rob587, May 10, 2013
    Replies:
    19
    Views:
    817
    msjaved
    Nov 8, 2013
  25. Ricky Martin 3124

    How to create Robots.txt Files?

    Ricky Martin 3124, Oct 31, 2013
    Replies:
    1
    Views:
    464
    Sayeri
    Oct 31, 2013
  26. yourarnav

    Googlebot can't access my site (Robot Txt)

    yourarnav, Sep 25, 2013
    Replies:
    4
    Views:
    1,005
    shakiba.proba
    Oct 9, 2013
  27. suganux

    Google Webmaster, URLs restricted by robots.txt

    suganux, Aug 13, 2013
    Replies:
    4
    Views:
    832
    suganux
    Aug 23, 2013
  28. PalakBhatt83

    Need Help with Robots.txt

    PalakBhatt83, Jul 24, 2013
    Replies:
    4
    Views:
    565
    blogdum
    Aug 12, 2013
  29. seoworld1234

    How to check robots.txt on site?

    seoworld1234, Apr 21, 2008 ... 2
    Replies:
    29
    Views:
    36,660
    Whitecap SEO
    Jul 23, 2013
  30. Goran Nikolov

    Robots.txt file can't be seen but SEOQuake says it exists?

    Goran Nikolov, Jul 11, 2013
    Replies:
    7
    Views:
    616
    MokricaIrina
    Jul 17, 2013
  31. prabhjot.singh

    How to Remove 404 Error?

    prabhjot.singh, Aug 26, 2012
    Replies:
    18
    Views:
    5,486
    Rakeshraj
    Jul 15, 2013
  32. HiSoC8Y

    Development site, don't want search engines...how?

    HiSoC8Y, Jul 12, 2013
    Replies:
    4
    Views:
    378
    ryan_uk
    Jul 13, 2013
  33. ntrdtr

    Blocked URL checking

    ntrdtr, Jun 11, 2013
    Replies:
    9
    Views:
    762
    Kyle Stester
    Jul 11, 2013
  34. w3seotraining
    Locked

    What is Robot.txt file?

    w3seotraining, Jul 3, 2013
    Replies:
    2
    Views:
    507
    Vincentas
    Jul 8, 2013
  35. tanmay.kamath1992

    Want to block other domain from crawling

    tanmay.kamath1992, Jun 22, 2013
    Replies:
    5
    Views:
    510
    Stephen Moyers
    Jul 4, 2013
  36. sam223

    Robots.txt - What is the proper format for a Crawl Delay for multiple user agents?

    sam223, Jun 29, 2013
    Replies:
    0
    Views:
    980
    sam223
    Jun 29, 2013
  37. beafriend

    When I search my blog on SE-I saw this-User-agent:Mediapartners-Google: * Disallow:

    beafriend, May 13, 2013
    Replies:
    3
    Views:
    1,418
    BobbyMcNiel
    Jun 3, 2013
  38. TiffanyJ.SSS

    Robots.txt security protection setup

    TiffanyJ.SSS, May 7, 2013
    Replies:
    2
    Views:
    449
    ryan_uk
    May 15, 2013
  39. sam2220

    Help regarding Robot.txt

    sam2220, Mar 19, 2013
    Replies:
    3
    Views:
    844
    topcoder
    May 14, 2013
  40. Tony Brar

    Stop Bing from indexing my website

    Tony Brar, Apr 22, 2013
    Replies:
    10
    Views:
    1,266
    Revelations-Decoder
    May 5, 2013
  41. Devtard

    How to prevent robots.txt from being indexed?

    Devtard, Apr 17, 2013
    Replies:
    5
    Views:
    582
    ryan_uk
    May 3, 2013
  42. Sanjeev SIngh

    IS robots.txt is safe for site?

    Sanjeev SIngh, Feb 21, 2013
    Replies:
    8
    Views:
    1,094
    blogdum
    Apr 25, 2013
  43. Devtard

    How to allow crawling a URL with a certain parameter

    Devtard, Apr 16, 2013
    Replies:
    1
    Views:
    400
    Devtard
    Apr 17, 2013
  44. Mr.Dog

    How Do I Ban Google From Seeing A Folder?

    Mr.Dog, Jan 16, 2013
    Replies:
    8
    Views:
    1,132
    MayurKawale
    Mar 29, 2013
  45. viveknathr

    Block a URL using Robots.txt

    viveknathr, Mar 24, 2013
    Replies:
    3
    Views:
    955
    indyonline
    Mar 24, 2013
  46. zeel_shah

    right and default format of robots.txt

    zeel_shah, Mar 13, 2013
    Replies:
    2
    Views:
    1,012
    ryan_uk
    Mar 14, 2013
  47. msaeedkk

    disallow:/?

    msaeedkk, Mar 7, 2013
    Replies:
    6
    Views:
    1,163
    ryan_uk
    Mar 14, 2013
  48. albertthomas

    Need Suggestion for Robots.txt

    albertthomas, May 23, 2012
    Replies:
    7
    Views:
    3,020
    WebNet Host
    Feb 21, 2013
  49. cartoonbabyboy

    Will Google Crawl Links On A Robot Disallowed Page?

    cartoonbabyboy, Feb 7, 2013
    Replies:
    3
    Views:
    1,075
    cartoonbabyboy
    Feb 9, 2013
  50. skylinkseo

    What its means in Robot.txt file Please help me

    skylinkseo, Jan 11, 2013
    Replies:
    1
    Views:
    1,206
    DaffodilSW
    Jan 17, 2013
Showing threads 1 to 50 of 861

Thread Display Options

Loading...
Page 1 of 18