robots.txt

Find help with setting up your robots.txt file on your web site.

Page 13 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. saadhome2

    What Is Use Of Robot Text File.

    saadhome2, Mar 24, 2010
    Replies:
    13
    Views:
    2,524
    seo-india
    Aug 27, 2010
  2. kevoh99

    Is robots.txt really safe?

    kevoh99, Jul 17, 2010
    Replies:
    12
    Views:
    3,009
    seo.lover
    Aug 29, 2010
  3. solution9

    how i can force search engine crawler to crawl my site ?

    solution9, Aug 15, 2010
    Replies:
    8
    Views:
    3,779
    seo.lover
    Aug 29, 2010
  4. eniva

    Best Robots.txt For Wordpress Installed Blogs

    eniva, Sep 23, 2008
    Replies:
    12
    Views:
    11,084
    seo.lover
    Aug 29, 2010
  5. jitendraag

    robots.txt for recruiting SEOs

    jitendraag, Aug 25, 2010
    Replies:
    3
    Views:
    1,411
    manish.chauhan
    Aug 30, 2010
  6. mikstroa

    I have a doubt in robots.txt - need help

    mikstroa, Aug 30, 2010
    Replies:
    4
    Views:
    1,318
    mikstroa
    Aug 30, 2010
  7. yulsa

    Do search crawlers see robots.txt or redirects first?

    yulsa, Aug 28, 2010
    Replies:
    3
    Views:
    1,679
    yulsa
    Aug 31, 2010
  8. Fracisc

    NoFollow links from robots.txt

    Fracisc, Aug 15, 2010
    Replies:
    7
    Views:
    1,788
    ravishanker
    Sep 1, 2010
  9. .TIEU

    All except One - Robots.txt

    .TIEU, Aug 28, 2010
    Replies:
    8
    Views:
    5,182
    .TIEU
    Sep 2, 2010
  10. sponge.nika

    I need a robots.txt to allow converted urls

    sponge.nika, Sep 4, 2010
    Replies:
    1
    Views:
    1,300
    digitalpoint
    Sep 4, 2010
  11. blankeyecue

    robot.txt help

    blankeyecue, Jul 27, 2010
    Replies:
    4
    Views:
    1,794
    The_Hatta
    Sep 20, 2010
  12. adwaitk007

    I need guidance! For good robot.txt| All Gurus are invited!

    adwaitk007, Sep 12, 2010
    Replies:
    6
    Views:
    1,703
    The_Hatta
    Sep 20, 2010
  13. aka_le_Mulder

    Optimal 'robots.txt' file for Joomla! 1.5?

    aka_le_Mulder, Jan 5, 2010
    Replies:
    1
    Views:
    4,389
    Sketch5646
    Sep 28, 2010
  14. life31

    DotBot!!! Is it of any use!!! Allow or Disallow???

    life31, Jun 21, 2009
    Replies:
    1
    Views:
    2,209
    Sketch5646
    Sep 28, 2010
  15. thomasonk

    Blocking DNS aliases/CNAMES

    thomasonk, Mar 9, 2007
    Replies:
    1
    Views:
    1,816
    Sketch5646
    Sep 28, 2010
  16. peterzhou

    The robots.txt file in my site does not work?

    peterzhou, Aug 5, 2010
    Replies:
    3
    Views:
    1,693
    Sudhaa
    Sep 28, 2010
  17. cahyoaja

    my robot.txt

    cahyoaja, Mar 23, 2010
    Replies:
    3
    Views:
    1,801
    seoguy5
    Oct 3, 2010
  18. Sammy04

    Question about blocking a link

    Sammy04, Mar 18, 2010
    Replies:
    2
    Views:
    1,643
    seoguy5
    Oct 3, 2010
  19. Alevoor

    I want to handpick Robots, spiders or crawlers, But How?

    Alevoor, Apr 22, 2010
    Replies:
    2
    Views:
    1,541
    seoguy5
    Oct 3, 2010
  20. thetaxclub

    Robots help - The Tax Club

    thetaxclub, Dec 6, 2006
    Replies:
    2
    Views:
    1,930
    manish.chauhan
    Oct 6, 2010
  21. alokseo

    Anybody tell me about robots.txt? hwo to usefull for seo?

    alokseo, Jun 18, 2010
    Replies:
    19
    Views:
    3,339
    Prateekgupta
    Oct 17, 2010
  22. ahmadlucky

    somebody plz solve my problem

    ahmadlucky, Sep 28, 2010
    Replies:
    8
    Views:
    2,316
    ahmadlucky
    Oct 17, 2010
  23. cahyoaja

    robot.txt. on blogspot

    cahyoaja, Oct 22, 2010
    Replies:
    1
    Views:
    1,562
    manish.chauhan
    Oct 24, 2010
  24. relkeel

    Mapped subdomains and robots.txt

    relkeel, Oct 7, 2010
    Replies:
    2
    Views:
    1,794
    manish.chauhan
    Oct 24, 2010
  25. tshirtdeal

    Objects Filename Restricted By Google Robots?

    tshirtdeal, Apr 14, 2009
    Replies:
    2
    Views:
    2,203
    manish.chauhan
    Oct 25, 2010
  26. Aragorn

    Separate page for mobile crawlers

    Aragorn, Oct 4, 2007
    Replies:
    2
    Views:
    2,569
    manish.chauhan
    Oct 25, 2010
  27. hectoth

    Robots.txt

    hectoth, Oct 7, 2010
    Replies:
    12
    Views:
    2,538
    Pronto! SEO
    Oct 28, 2010
  28. avatarmaddy

    Is really the contents provided by robots are meaningful ?

    avatarmaddy, Oct 27, 2010
    Replies:
    4
    Views:
    1,816
    avatarmaddy
    Oct 31, 2010
  29. sorcha

    Can I limt a crawler to the root fodler with a single command?

    sorcha, Nov 1, 2010
    Replies:
    1
    Views:
    1,467
    AtomicPages
    Nov 3, 2010
  30. daliah

    my robot file doesnot work

    daliah, Oct 15, 2010
    Replies:
    9
    Views:
    2,107
    lyco
    Nov 14, 2010
  31. nuraddiansyah

    PR3 blog search advertisers

    nuraddiansyah, Nov 15, 2010
    Replies:
    0
    Views:
    1,742
    nuraddiansyah
    Nov 15, 2010
  32. duffer77

    Block specific part of the site

    duffer77, Nov 22, 2010
    Replies:
    2
    Views:
    1,631
    The_Hatta
    Nov 24, 2010
  33. fokerss

    what is agent for bing ,blekko,bweezy?

    fokerss, Nov 17, 2010
    Replies:
    1
    Views:
    1,591
    underground-stockholm
    Nov 30, 2010
  34. vincent87

    robots.txt for smf forum

    vincent87, Dec 8, 2010
    Replies:
    0
    Views:
    1,971
    vincent87
    Dec 8, 2010
  35. scott.allerdice

    robots.txt for https

    scott.allerdice, Dec 13, 2010
    Replies:
    4
    Views:
    1,949
    scott.allerdice
    Dec 19, 2010
  36. jeel2008

    whats the root?

    jeel2008, Oct 22, 2010
    Replies:
    2
    Views:
    1,795
    computerlaptop
    Dec 23, 2010
  37. Woodcs82

    POLL: Do Large Websites your robots.txt ?

    Woodcs82, Nov 17, 2010
    Replies:
    1
    Views:
    1,713
    Pablo Peillard
    Dec 25, 2010
  38. pradeep.balua

    how to install a robots.txt for a wordpress blog ?

    pradeep.balua, Jan 22, 2008
    Replies:
    11
    Views:
    4,861
    ayushi infotech
    Dec 28, 2010
  39. articleshop4all

    Robots.txt Please Check mine

    articleshop4all, Oct 30, 2010
    Replies:
    9
    Views:
    2,399
    reihaan
    Dec 30, 2010
  40. ayushi infotech

    Pls rectify my problem in Robots.txt

    ayushi infotech, Dec 28, 2010
    Replies:
    1
    Views:
    1,841
    techvivek
    Dec 31, 2010
  41. DoDo Me

    Help! I want a robots.txt allow only root and disallow all others

    DoDo Me, Aug 30, 2010
    Replies:
    4
    Views:
    5,593
    techvivek
    Dec 31, 2010
  42. penpebso

    Should i continue to block robots from redirect link?

    penpebso, Jan 11, 2011
    Replies:
    0
    Views:
    1,829
    penpebso
    Jan 11, 2011
  43. lxguy

    How to block some particular urls in my robots.txt?

    lxguy, Jan 16, 2011
    Replies:
    1
    Views:
    6,081
    nit Green
    Jan 19, 2011
  44. philipseocc

    Can robotx.txt block specific IPs/domain names?

    philipseocc, Jan 25, 2011
    Replies:
    7
    Views:
    2,855
    philipseocc
    Feb 1, 2011
  45. n00bl3t

    Google webcache banned by robots.txt

    n00bl3t, Nov 25, 2010
    Replies:
    3
    Views:
    2,742
    gmendoza_0101
    Feb 7, 2011
  46. ljk6802

    Layman's guide to robots.txt (and sitemap)

    ljk6802, Jan 24, 2011
    Replies:
    8
    Views:
    2,794
    ljk6802
    Feb 7, 2011
  47. illianafrazer

    How can i block Google bot alone

    illianafrazer, Feb 2, 2011
    Replies:
    4
    Views:
    2,696
    fsdnetwork
    Feb 9, 2011
  48. gravy834

    Robots.txt vs <meta name="robots" content="noindex,nofollow">

    gravy834, Mar 2, 2009
    Replies:
    12
    Views:
    8,860
    fsdnetwork
    Feb 9, 2011
  49. Pablo Peillard

    My Website is being indexed a little too much

    Pablo Peillard, Dec 25, 2010
    Replies:
    6
    Views:
    2,702
    fsdnetwork
    Feb 9, 2011
  50. Woodcs82

    Is Robots.txt better than Robots in META?

    Woodcs82, Jun 10, 2010 ... 2
    Replies:
    24
    Views:
    7,325
    fsdnetwork
    Feb 9, 2011
Showing threads 601 to 650 of 911

Thread Display Options

Loading...
Page 13 of 19