robots.txt

Find help with setting up your robots.txt file on your web site.

Page 17 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. Haki88

    securing cms and ipb forum

    Haki88, Nov 10, 2012
    Replies:
    5
    Views:
    3,194
    Vvardenfell
    Dec 12, 2012
  2. riteshvyas52

    robots.txt in magento ?

    riteshvyas52, Dec 13, 2012
    Replies:
    1
    Views:
    2,048
    leo29
    Dec 13, 2012
  3. Ar!yaN

    Robots.txt Error in Google Webmaster

    Ar!yaN, Dec 15, 2012
    Replies:
    3
    Views:
    1,981
    mymake
    Dec 24, 2012
  4. Stephen H.Cummings

    robots.txt for WHMCS

    Stephen H.Cummings, Dec 26, 2012
    Replies:
    1
    Views:
    5,734
    selectaupairs
    Dec 26, 2012
  5. HomeRun

    Advantages of robots.txt

    HomeRun, Feb 24, 2008 ... 2
    Replies:
    26
    Views:
    18,424
    agitetech
    Dec 27, 2012
  6. towfiq71

    i use this as my robots.txt file. is it right?

    towfiq71, Nov 24, 2012
    Replies:
    4
    Views:
    2,496
    agitetech
    Dec 27, 2012
  7. Deepak_Kotian

    Robot.txt beginner

    Deepak_Kotian, Nov 1, 2012
    Replies:
    6
    Views:
    3,028
    agitetech
    Dec 27, 2012
  8. geniusoptimizer

    What should be robots.txt file for these indexed urls??

    geniusoptimizer, Dec 17, 2012
    Replies:
    4
    Views:
    2,034
    icool89
    Jan 2, 2013
  9. skylinkseo

    What its means in Robot.txt file Please help me

    skylinkseo, Jan 11, 2013
    Replies:
    1
    Views:
    2,171
    DaffodilSW
    Jan 17, 2013
  10. cartoonbabyboy

    Will Google Crawl Links On A Robot Disallowed Page?

    cartoonbabyboy, Feb 7, 2013
    Replies:
    3
    Views:
    2,008
    cartoonbabyboy
    Feb 9, 2013
  11. albertthomas

    Need Suggestion for Robots.txt

    albertthomas, May 23, 2012
    Replies:
    7
    Views:
    4,128
    WebNet Host
    Feb 21, 2013
  12. msaeedkk

    disallow:/?

    msaeedkk, Mar 7, 2013
    Replies:
    6
    Views:
    2,419
    ryan_uk
    Mar 14, 2013
  13. zeel_shah

    right and default format of robots.txt

    zeel_shah, Mar 13, 2013
    Replies:
    2
    Views:
    1,979
    ryan_uk
    Mar 14, 2013
  14. viveknathr

    Block a URL using Robots.txt

    viveknathr, Mar 24, 2013
    Replies:
    3
    Views:
    1,981
    indyonline
    Mar 24, 2013
  15. Mr.Dog

    How Do I Ban Google From Seeing A Folder?

    Mr.Dog, Jan 16, 2013
    Replies:
    8
    Views:
    2,373
    MayurKawale
    Mar 29, 2013
  16. Devtard

    How to allow crawling a URL with a certain parameter

    Devtard, Apr 16, 2013
    Replies:
    1
    Views:
    1,210
    Devtard
    Apr 17, 2013
  17. Sanjeev SIngh

    IS robots.txt is safe for site?

    Sanjeev SIngh, Feb 21, 2013
    Replies:
    8
    Views:
    2,376
    blogdum
    Apr 25, 2013
  18. Devtard

    How to prevent robots.txt from being indexed?

    Devtard, Apr 17, 2013
    Replies:
    5
    Views:
    1,642
    ryan_uk
    May 3, 2013
  19. Tony Brar

    Stop Bing from indexing my website

    Tony Brar, Apr 22, 2013
    Replies:
    10
    Views:
    5,790
    Revelations-Decoder
    May 5, 2013
  20. sam2220

    Help regarding Robot.txt

    sam2220, Mar 19, 2013
    Replies:
    3
    Views:
    1,936
    topcoder
    May 14, 2013
  21. TiffanyJ.SSS

    Robots.txt security protection setup

    TiffanyJ.SSS, May 7, 2013
    Replies:
    2
    Views:
    2,840
    ryan_uk
    May 15, 2013
  22. beafriend

    When I search my blog on SE-I saw this-User-agent:Mediapartners-Google: * Disallow:

    beafriend, May 13, 2013
    Replies:
    3
    Views:
    4,915
    BobbyMcNiel
    Jun 3, 2013
  23. sam223

    Robots.txt - What is the proper format for a Crawl Delay for multiple user agents?

    sam223, Jun 29, 2013
    Replies:
    0
    Views:
    4,386
    sam223
    Jun 29, 2013
  24. tanmay.kamath1992

    Want to block other domain from crawling

    tanmay.kamath1992, Jun 22, 2013
    Replies:
    5
    Views:
    3,052
    Stephen Moyers
    Jul 4, 2013
  25. w3seotraining
    Locked

    What is Robot.txt file?

    w3seotraining, Jul 3, 2013
    Replies:
    2
    Views:
    2,940
    Vincentas
    Jul 8, 2013
  26. ntrdtr

    Blocked URL checking

    ntrdtr, Jun 11, 2013
    Replies:
    9
    Views:
    3,839
    Kyle Stester
    Jul 11, 2013
  27. HiSoC8Y

    Development site, don't want search engines...how?

    HiSoC8Y, Jul 12, 2013
    Replies:
    4
    Views:
    2,930
    ryan_uk
    Jul 13, 2013
  28. prabhjot.singh

    How to Remove 404 Error?

    prabhjot.singh, Aug 26, 2012
    Replies:
    18
    Views:
    11,956
    Rakeshraj
    Jul 15, 2013
  29. Goran Nikolov

    Robots.txt file can't be seen but SEOQuake says it exists?

    Goran Nikolov, Jul 11, 2013
    Replies:
    7
    Views:
    6,037
    MokricaIrina
    Jul 17, 2013
  30. PalakBhatt83

    Need Help with Robots.txt

    PalakBhatt83, Jul 24, 2013
    Replies:
    4
    Views:
    5,745
    blogdum
    Aug 12, 2013
  31. suganux

    Google Webmaster, URLs restricted by robots.txt

    suganux, Aug 13, 2013
    Replies:
    4
    Views:
    6,362
    suganux
    Aug 23, 2013
  32. yourarnav

    Googlebot can't access my site (Robot Txt)

    yourarnav, Sep 25, 2013
    Replies:
    4
    Views:
    6,721
    shakiba.proba
    Oct 9, 2013
  33. Ricky Martin 3124

    How to create Robots.txt Files?

    Ricky Martin 3124, Oct 31, 2013
    Replies:
    1
    Views:
    5,826
    Sayeri
    Oct 31, 2013
  34. rob587

    Would a robot.txt fix this SEO issue?

    rob587, May 10, 2013
    Replies:
    19
    Views:
    11,188
    msjaved
    Nov 8, 2013
  35. chadhaajay

    Please review my robots.txt code

    chadhaajay, Oct 10, 2013
    Replies:
    2
    Views:
    8,266
    makeit easy
    Nov 17, 2013
  36. Vel

    I need help on how to Noindex categories, tags, author, archive

    Vel, Jul 25, 2013
    Replies:
    6
    Views:
    8,876
    tymsh
    Dec 9, 2013
  37. Sunil Keshari

    Robots.txt code

    Sunil Keshari, Jan 9, 2014
    Replies:
    3
    Views:
    10,011
    ashokdhan
    Jan 14, 2014
  38. vatu

    Robots.txt File Help

    vatu, Jan 15, 2014
    Replies:
    8
    Views:
    8,769
    alsgroup
    Jan 23, 2014
  39. SebastienD

    Dissalow folder with the same name as allowed folder

    SebastienD, Jan 15, 2014
    Replies:
    2
    Views:
    9,432
    Nette
    Feb 2, 2014
  40. manish_khanna

    Things you should be doing to block bad BOTS.

    manish_khanna, Feb 13, 2014
    Replies:
    0
    Views:
    8,639
    manish_khanna
    Feb 13, 2014
  41. cathandrew

    Is it necessary to specify sitemap.xml in robots.txt file

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    8,892
    Ehteshamshaikh
    Feb 27, 2014
  42. bright776

    how to disallow a folder from being indexed

    bright776, Feb 1, 2014
    Replies:
    3
    Views:
    9,158
    Ehteshamshaikh
    Mar 7, 2014
  43. garry420

    I have at-last found site that creates xml file of https sites

    garry420, Mar 8, 2014
    Replies:
    0
    Views:
    8,206
    garry420
    Mar 8, 2014
  44. James101

    What is robot.txt file?

    James101, Apr 1, 2014
    Replies:
    4
    Views:
    9,597
    John Dave
    Apr 2, 2014
  45. jiangnancun

    How can i find those website in which robots.txt disallow google spider?

    jiangnancun, May 13, 2014
    Replies:
    2
    Views:
    9,282
    neroux
    May 13, 2014
  46. imaginexhacked

    Blocking auto generated pages with robots.txt

    imaginexhacked, Jun 9, 2014
    Replies:
    1
    Views:
    8,702
    O-D-T
    Jun 9, 2014
  47. BabyDoll123

    Magento Robots.txt file

    BabyDoll123, Jun 12, 2014
    Replies:
    3
    Views:
    9,929
    Nigel Lew
    Jun 14, 2014
  48. mjfnd

    how to dissalow an image?

    mjfnd, Apr 29, 2014
    Replies:
    1
    Views:
    8,781
    BabyDoll123
    Jun 19, 2014
  49. cathandrew

    How to clear these errors in robots.txt?

    cathandrew, Feb 25, 2014
    Replies:
    1
    Views:
    9,081
    BabyDoll123
    Jun 19, 2014
  50. David Lowe

    Can I add specific urls to robots.txt to stop google indexing them?

    David Lowe, Jun 25, 2013
    Replies:
    2
    Views:
    9,962
    BabyDoll123
    Jun 19, 2014
Showing threads 801 to 850 of 911

Thread Display Options

Loading...
Page 17 of 19