Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 2 of 18
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. David Lowe

    Can I add specific urls to robots.txt to stop google indexing them?

    David Lowe, Jun 25, 2013
    Replies:
    2
    Views:
    9,396
    BabyDoll123
    Jun 19, 2014
  2. cathandrew

    How to clear these errors in robots.txt?

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    8,509
    BabyDoll123
    Jun 19, 2014
  3. mjfnd

    how to dissalow an image?

    mjfnd, Apr 29, 2014
    Replies:
    1
    Views:
    8,239
    BabyDoll123
    Jun 19, 2014
  4. BabyDoll123

    Magento Robots.txt file

    BabyDoll123, Jun 12, 2014
    Replies:
    3
    Views:
    9,154
    Nigel Lew
    Jun 14, 2014
  5. imaginexhacked

    Blocking auto generated pages with robots.txt

    imaginexhacked, Jun 9, 2014
    Replies:
    1
    Views:
    8,135
    O-D-T
    Jun 9, 2014
  6. jiangnancun

    How can i find those website in which robots.txt disallow google spider?

    jiangnancun, May 13, 2014
    Replies:
    2
    Views:
    8,570
    neroux
    May 13, 2014
  7. James101

    What is robot.txt file?

    James101, Apr 1, 2014
    Replies:
    4
    Views:
    8,957
    John Dave
    Apr 2, 2014
  8. garry420

    I have at-last found site that creates xml file of https sites

    garry420, Mar 8, 2014
    Replies:
    0
    Views:
    7,726
    garry420
    Mar 8, 2014
  9. bright776

    how to disallow a folder from being indexed

    bright776, Feb 1, 2014
    Replies:
    3
    Views:
    8,482
    Ehteshamshaikh
    Mar 7, 2014
  10. cathandrew

    Is it necessary to specify sitemap.xml in robots.txt file

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    8,354
    Ehteshamshaikh
    Feb 27, 2014
  11. manish_khanna

    Things you should be doing to block bad BOTS.

    manish_khanna, Feb 13, 2014
    Replies:
    0
    Views:
    8,000
    manish_khanna
    Feb 13, 2014
  12. SebastienD

    Dissalow folder with the same name as allowed folder

    SebastienD, Jan 15, 2014
    Replies:
    2
    Views:
    8,864
    Nette
    Feb 2, 2014
  13. vatu

    Robots.txt File Help

    vatu, Jan 15, 2014
    Replies:
    8
    Views:
    8,042
    alsgroup
    Jan 23, 2014
  14. Sunil Keshari

    Robots.txt code

    Sunil Keshari, Jan 9, 2014
    Replies:
    3
    Views:
    9,508
    ashokdhan
    Jan 14, 2014
  15. Vel

    I need help on how to Noindex categories, tags, author, archive

    Vel, Jul 25, 2013
    Replies:
    6
    Views:
    8,312
    tymsh
    Dec 9, 2013
  16. chadhaajay

    Please review my robots.txt code

    chadhaajay, Oct 10, 2013
    Replies:
    2
    Views:
    7,797
    makeit easy
    Nov 17, 2013
  17. rob587

    Would a robot.txt fix this SEO issue?

    rob587, May 10, 2013
    Replies:
    19
    Views:
    9,950
    msjaved
    Nov 8, 2013
  18. Ricky Martin 3124

    How to create Robots.txt Files?

    Ricky Martin 3124, Oct 31, 2013
    Replies:
    1
    Views:
    5,290
    Sayeri
    Oct 31, 2013
  19. yourarnav

    Googlebot can't access my site (Robot Txt)

    yourarnav, Sep 25, 2013
    Replies:
    4
    Views:
    6,223
    shakiba.proba
    Oct 9, 2013
  20. suganux

    Google Webmaster, URLs restricted by robots.txt

    suganux, Aug 13, 2013
    Replies:
    4
    Views:
    5,696
    suganux
    Aug 23, 2013
  21. PalakBhatt83

    Need Help with Robots.txt

    PalakBhatt83, Jul 24, 2013
    Replies:
    4
    Views:
    5,257
    blogdum
    Aug 12, 2013
  22. Goran Nikolov

    Robots.txt file can't be seen but SEOQuake says it exists?

    Goran Nikolov, Jul 11, 2013
    Replies:
    7
    Views:
    5,472
    MokricaIrina
    Jul 17, 2013
  23. prabhjot.singh

    How to Remove 404 Error?

    prabhjot.singh, Aug 26, 2012
    Replies:
    18
    Views:
    10,414
    Rakeshraj
    Jul 15, 2013
  24. HiSoC8Y

    Development site, don't want search engines...how?

    HiSoC8Y, Jul 12, 2013
    Replies:
    4
    Views:
    2,452
    ryan_uk
    Jul 13, 2013
  25. ntrdtr

    Blocked URL checking

    ntrdtr, Jun 11, 2013
    Replies:
    9
    Views:
    3,018
    Kyle Stester
    Jul 11, 2013
  26. w3seotraining
    Locked

    What is Robot.txt file?

    w3seotraining, Jul 3, 2013
    Replies:
    2
    Views:
    2,485
    Vincentas
    Jul 8, 2013
  27. tanmay.kamath1992

    Want to block other domain from crawling

    tanmay.kamath1992, Jun 22, 2013
    Replies:
    5
    Views:
    2,539
    Stephen Moyers
    Jul 4, 2013
  28. sam223

    Robots.txt - What is the proper format for a Crawl Delay for multiple user agents?

    sam223, Jun 29, 2013
    Replies:
    0
    Views:
    3,762
    sam223
    Jun 29, 2013
  29. beafriend

    When I search my blog on SE-I saw this-User-agent:Mediapartners-Google: * Disallow:

    beafriend, May 13, 2013
    Replies:
    3
    Views:
    3,937
    BobbyMcNiel
    Jun 3, 2013
  30. TiffanyJ.SSS

    Robots.txt security protection setup

    TiffanyJ.SSS, May 7, 2013
    Replies:
    2
    Views:
    2,375
    ryan_uk
    May 15, 2013
  31. sam2220

    Help regarding Robot.txt

    sam2220, Mar 19, 2013
    Replies:
    3
    Views:
    1,426
    topcoder
    May 14, 2013
  32. Tony Brar

    Stop Bing from indexing my website

    Tony Brar, Apr 22, 2013
    Replies:
    10
    Views:
    3,582
    Revelations-Decoder
    May 5, 2013
  33. Devtard

    How to prevent robots.txt from being indexed?

    Devtard, Apr 17, 2013
    Replies:
    5
    Views:
    1,104
    ryan_uk
    May 3, 2013
  34. Sanjeev SIngh

    IS robots.txt is safe for site?

    Sanjeev SIngh, Feb 21, 2013
    Replies:
    8
    Views:
    1,655
    blogdum
    Apr 25, 2013
  35. Devtard

    How to allow crawling a URL with a certain parameter

    Devtard, Apr 16, 2013
    Replies:
    1
    Views:
    829
    Devtard
    Apr 17, 2013
  36. Mr.Dog

    How Do I Ban Google From Seeing A Folder?

    Mr.Dog, Jan 16, 2013
    Replies:
    8
    Views:
    1,670
    MayurKawale
    Mar 29, 2013
  37. viveknathr

    Block a URL using Robots.txt

    viveknathr, Mar 24, 2013
    Replies:
    3
    Views:
    1,407
    indyonline
    Mar 24, 2013
  38. zeel_shah

    right and default format of robots.txt

    zeel_shah, Mar 13, 2013
    Replies:
    2
    Views:
    1,457
    ryan_uk
    Mar 14, 2013
  39. msaeedkk

    disallow:/?

    msaeedkk, Mar 7, 2013
    Replies:
    6
    Views:
    1,699
    ryan_uk
    Mar 14, 2013
  40. albertthomas

    Need Suggestion for Robots.txt

    albertthomas, May 23, 2012
    Replies:
    7
    Views:
    3,421
    WebNet Host
    Feb 21, 2013
  41. cartoonbabyboy

    Will Google Crawl Links On A Robot Disallowed Page?

    cartoonbabyboy, Feb 7, 2013
    Replies:
    3
    Views:
    1,473
    cartoonbabyboy
    Feb 9, 2013
  42. skylinkseo

    What its means in Robot.txt file Please help me

    skylinkseo, Jan 11, 2013
    Replies:
    1
    Views:
    1,636
    DaffodilSW
    Jan 17, 2013
  43. geniusoptimizer

    What should be robots.txt file for these indexed urls??

    geniusoptimizer, Dec 17, 2012
    Replies:
    4
    Views:
    1,481
    icool89
    Jan 2, 2013
  44. Deepak_Kotian

    Robot.txt beginner

    Deepak_Kotian, Nov 1, 2012
    Replies:
    6
    Views:
    2,378
    agitetech
    Dec 27, 2012
  45. towfiq71

    i use this as my robots.txt file. is it right?

    towfiq71, Nov 24, 2012
    Replies:
    4
    Views:
    1,795
    agitetech
    Dec 27, 2012
  46. HomeRun

    Advantages of robots.txt

    HomeRun, Feb 24, 2008 ... 2
    Replies:
    26
    Views:
    15,674
    agitetech
    Dec 27, 2012
  47. Stephen H.Cummings

    robots.txt for WHMCS

    Stephen H.Cummings, Dec 26, 2012
    Replies:
    1
    Views:
    3,153
    selectaupairs
    Dec 26, 2012
  48. Ar!yaN

    Robots.txt Error in Google Webmaster

    Ar!yaN, Dec 15, 2012
    Replies:
    3
    Views:
    1,487
    mymake
    Dec 24, 2012
  49. riteshvyas52

    robots.txt in magento ?

    riteshvyas52, Dec 13, 2012
    Replies:
    1
    Views:
    1,553
    leo29
    Dec 13, 2012
  50. Haki88

    securing cms and ipb forum

    Haki88, Nov 10, 2012
    Replies:
    5
    Views:
    2,447
    Vvardenfell
    Dec 12, 2012
Showing threads 51 to 100 of 900

Thread Display Options

Loading...
Page 2 of 18