Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 1 of 18
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. Mrinal_Roy

    Robots.txt is blocking my labels

    Mrinal_Roy, Feb 5, 2015
    Replies:
    1
    Views:
    537
    vgopalsujith
    Feb 5, 2015
  2. Prasseo

    Will robots file help to cache the content in the web page

    Prasseo, Jan 12, 2015
    Replies:
    1
    Views:
    443
    Sycrid
    Jan 18, 2015
  3. scott2342

    How do I unblock my robots.txt

    scott2342, Nov 24, 2014
    Replies:
    1
    Views:
    369
    Sagar More
    Jan 16, 2015
  4. wayne_hoton

    How to create robots.txt for bookmarking website. Is it necessary?

    wayne_hoton, Jan 11, 2015
    Replies:
    1
    Views:
    202
    wayne_hoton
    Jan 14, 2015
  5. Prasseo

    How to create a robots file for PHP webpage to crawl

    Prasseo, Jan 12, 2015
    Replies:
    0
    Views:
    160
    Prasseo
    Jan 12, 2015
  6. myfastrupee

    Robot.txt file not found on my blog

    myfastrupee, Jun 21, 2013
    Replies:
    3
    Views:
    2,494
    pushpak479
    Dec 1, 2014
  7. gree124

    robots.txt please check

    gree124, Nov 27, 2014
    Replies:
    3
    Views:
    291
    Naina S
    Dec 1, 2014
  8. Jaden Alvin

    Webmaster Blocked URL's Understanding Issue

    Jaden Alvin, Jul 9, 2014 ... 2
    Replies:
    20
    Views:
    2,966
    Jaden Alvin
    Nov 24, 2014
  9. Olvia

    can give an example of a good robots.txt to wordpress

    Olvia, Nov 13, 2014
    Replies:
    0
    Views:
    412
    Olvia
    Nov 13, 2014
  10. Angelabailey

    Robots.txt

    Angelabailey, Nov 15, 2013
    Replies:
    16
    Views:
    2,274
    lisagray304
    Oct 28, 2014
  11. aleale

    Is my robots.txt working fine?

    aleale, Sep 10, 2014
    Replies:
    2
    Views:
    2,697
    maddenitrous
    Oct 3, 2014
  12. gvinaykumar

    How to block a specific folder in robots.txt file?

    gvinaykumar, Jul 17, 2014
    Replies:
    0
    Views:
    1,430
    gvinaykumar
    Jul 17, 2014
  13. linkladdy

    Error - URLs restricted by robots.txt errors How Fix IT

    linkladdy, Jul 2, 2014
    Replies:
    1
    Views:
    1,473
    evildon
    Jul 2, 2014
  14. David Lowe

    Can I add specific urls to robots.txt to stop google indexing them?

    David Lowe, Jun 25, 2013
    Replies:
    2
    Views:
    2,452
    BabyDoll123
    Jun 19, 2014
  15. cathandrew

    How to clear these errors in robots.txt?

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    1,518
    BabyDoll123
    Jun 19, 2014
  16. mjfnd

    how to dissalow an image?

    mjfnd, Apr 29, 2014
    Replies:
    1
    Views:
    1,468
    BabyDoll123
    Jun 19, 2014
  17. BabyDoll123

    Magento Robots.txt file

    BabyDoll123, Jun 12, 2014
    Replies:
    3
    Views:
    1,620
    Nigel Lew
    Jun 14, 2014
  18. imaginexhacked

    Blocking auto generated pages with robots.txt

    imaginexhacked, Jun 9, 2014
    Replies:
    1
    Views:
    1,433
    O-D-T
    Jun 9, 2014
  19. jiangnancun

    How can i find those website in which robots.txt disallow google spider?

    jiangnancun, May 13, 2014
    Replies:
    2
    Views:
    1,453
    neroux
    May 13, 2014
  20. James101

    What is robot.txt file?

    James101, Apr 1, 2014
    Replies:
    4
    Views:
    1,787
    John Dave
    Apr 2, 2014
  21. garry420

    I have at-last found site that creates xml file of https sites

    garry420, Mar 8, 2014
    Replies:
    0
    Views:
    1,282
    garry420
    Mar 8, 2014
  22. bright776

    how to disallow a folder from being indexed

    bright776, Feb 1, 2014
    Replies:
    3
    Views:
    1,444
    Ehteshamshaikh
    Mar 7, 2014
  23. cathandrew

    Is it necessary to specify sitemap.xml in robots.txt file

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    1,365
    Ehteshamshaikh
    Feb 27, 2014
  24. manish_khanna

    Things you should be doing to block bad BOTS.

    manish_khanna, Feb 13, 2014
    Replies:
    0
    Views:
    1,447
    manish_khanna
    Feb 13, 2014
  25. SebastienD

    Dissalow folder with the same name as allowed folder

    SebastienD, Jan 15, 2014
    Replies:
    2
    Views:
    1,817
    Nette
    Feb 2, 2014
  26. vatu

    Robots.txt File Help

    vatu, Jan 15, 2014
    Replies:
    8
    Views:
    1,585
    alsgroup
    Jan 23, 2014
  27. Sunil Keshari

    Robots.txt code

    Sunil Keshari, Jan 9, 2014
    Replies:
    3
    Views:
    1,971
    ashokdhan
    Jan 14, 2014
  28. Vel

    I need help on how to Noindex categories, tags, author, archive

    Vel, Jul 25, 2013
    Replies:
    6
    Views:
    1,965
    tymsh
    Dec 9, 2013
  29. chadhaajay

    Please review my robots.txt code

    chadhaajay, Oct 10, 2013
    Replies:
    2
    Views:
    1,532
    makeit easy
    Nov 17, 2013
  30. pradip200

    Robots.txt for SMF Forum

    pradip200, Mar 23, 2013
    Replies:
    3
    Views:
    3,949
    donjajo
    Nov 9, 2013
  31. rob587

    Would a robot.txt fix this SEO issue?

    rob587, May 10, 2013
    Replies:
    19
    Views:
    1,023
    msjaved
    Nov 8, 2013
  32. Ricky Martin 3124

    How to create Robots.txt Files?

    Ricky Martin 3124, Oct 31, 2013
    Replies:
    1
    Views:
    571
    Sayeri
    Oct 31, 2013
  33. yourarnav

    Googlebot can't access my site (Robot Txt)

    yourarnav, Sep 25, 2013
    Replies:
    4
    Views:
    1,263
    shakiba.proba
    Oct 9, 2013
  34. suganux

    Google Webmaster, URLs restricted by robots.txt

    suganux, Aug 13, 2013
    Replies:
    4
    Views:
    964
    suganux
    Aug 23, 2013
  35. PalakBhatt83

    Need Help with Robots.txt

    PalakBhatt83, Jul 24, 2013
    Replies:
    4
    Views:
    680
    blogdum
    Aug 12, 2013
  36. seoworld1234

    How to check robots.txt on site?

    seoworld1234, Apr 21, 2008 ... 2
    Replies:
    28
    Views:
    38,782
    Whitecap SEO
    Jul 23, 2013
  37. Goran Nikolov

    Robots.txt file can't be seen but SEOQuake says it exists?

    Goran Nikolov, Jul 11, 2013
    Replies:
    7
    Views:
    766
    MokricaIrina
    Jul 17, 2013
  38. prabhjot.singh

    How to Remove 404 Error?

    prabhjot.singh, Aug 26, 2012
    Replies:
    18
    Views:
    5,736
    Rakeshraj
    Jul 15, 2013
  39. HiSoC8Y

    Development site, don't want search engines...how?

    HiSoC8Y, Jul 12, 2013
    Replies:
    4
    Views:
    487
    ryan_uk
    Jul 13, 2013
  40. ntrdtr

    Blocked URL checking

    ntrdtr, Jun 11, 2013
    Replies:
    9
    Views:
    904
    Kyle Stester
    Jul 11, 2013
  41. w3seotraining
    Locked

    What is Robot.txt file?

    w3seotraining, Jul 3, 2013
    Replies:
    2
    Views:
    624
    Vincentas
    Jul 8, 2013
  42. tanmay.kamath1992

    Want to block other domain from crawling

    tanmay.kamath1992, Jun 22, 2013
    Replies:
    5
    Views:
    620
    Stephen Moyers
    Jul 4, 2013
  43. sam223

    Robots.txt - What is the proper format for a Crawl Delay for multiple user agents?

    sam223, Jun 29, 2013
    Replies:
    0
    Views:
    1,209
    sam223
    Jun 29, 2013
  44. beafriend

    When I search my blog on SE-I saw this-User-agent:Mediapartners-Google: * Disallow:

    beafriend, May 13, 2013
    Replies:
    3
    Views:
    1,721
    BobbyMcNiel
    Jun 3, 2013
  45. TiffanyJ.SSS

    Robots.txt security protection setup

    TiffanyJ.SSS, May 7, 2013
    Replies:
    2
    Views:
    557
    ryan_uk
    May 15, 2013
  46. sam2220

    Help regarding Robot.txt

    sam2220, Mar 19, 2013
    Replies:
    3
    Views:
    958
    topcoder
    May 14, 2013
  47. Tony Brar

    Stop Bing from indexing my website

    Tony Brar, Apr 22, 2013
    Replies:
    10
    Views:
    1,582
    Revelations-Decoder
    May 5, 2013
  48. Devtard

    How to prevent robots.txt from being indexed?

    Devtard, Apr 17, 2013
    Replies:
    5
    Views:
    693
    ryan_uk
    May 3, 2013
  49. Sanjeev SIngh

    IS robots.txt is safe for site?

    Sanjeev SIngh, Feb 21, 2013
    Replies:
    8
    Views:
    1,207
    blogdum
    Apr 25, 2013
  50. Devtard

    How to allow crawling a URL with a certain parameter

    Devtard, Apr 16, 2013
    Replies:
    1
    Views:
    508
    Devtard
    Apr 17, 2013
Showing threads 1 to 50 of 868

Thread Display Options

Loading...
Page 1 of 18