1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 1 of 19
Sort by:
Title Start Date
Replies Views ↑
Last Message
  1. filmixpress
    Locked

    Robot

    filmixpress, Oct 28, 2008
    Replies:
    0
    Views:
    773
    filmixpress
    Oct 28, 2008
  2. warley

    Help Required

    warley, Mar 11, 2008
    Replies:
    2
    Views:
    778
    warley
    Mar 11, 2008
  3. Guitarkalle

    Is this correct?

    Guitarkalle, Jan 18, 2009
    Replies:
    3
    Views:
    782
    manish.chauhan
    Jan 20, 2009
  4. laxmi

    what to include ?

    laxmi, Jun 29, 2008
    Replies:
    2
    Views:
    806
    manish.chauhan
    Jul 1, 2008
  5. sandeeppandey

    Robot.Txt And RSS

    sandeeppandey, Jun 2, 2011
    Replies:
    4
    Views:
    806
    WishBone
    Jun 7, 2011
  6. jbperkins

    Need robots.txt Help

    jbperkins, Sep 24, 2008
    Replies:
    0
    Views:
    812
    jbperkins
    Sep 24, 2008
  7. intellg

    How to stop my all external link crawling?

    intellg, May 5, 2010
    Replies:
    1
    Views:
    816
    onlinesujeet
    May 18, 2010
  8. mybizhits

    What robots.txt for proxies?

    mybizhits, Jun 20, 2008
    Replies:
    1
    Views:
    831
    manish.chauhan
    Jun 21, 2008
  9. kiransarv

    robots.txt

    kiransarv, Oct 21, 2008
    Replies:
    0
    Views:
    847
    kiransarv
    Oct 21, 2008
  10. ysribharath

    am i correct ?

    ysribharath, Jun 10, 2010
    Replies:
    1
    Views:
    860
    manish.chauhan
    Jun 10, 2010
  11. Pixelrage

    how long to stop showing up in SERPS?

    Pixelrage, Oct 16, 2008
    Replies:
    2
    Views:
    865
    Pixelrage
    Oct 17, 2008
  12. makenoiz

    is /*? (astrisk question mark) the same as /? (question mark)

    makenoiz, Mar 3, 2010
    Replies:
    0
    Views:
    869
    makenoiz
    Mar 3, 2010
  13. inManila

    Regex and robots?

    inManila, Jan 16, 2009
    Replies:
    1
    Views:
    871
    manish.chauhan
    Jan 20, 2009
  14. TheMrAnderson

    Excluding a page and all spiders except Google

    TheMrAnderson, Oct 20, 2008
    Replies:
    0
    Views:
    873
    TheMrAnderson
    Oct 20, 2008
  15. iMax

    www and non-www crawl by bots

    iMax, Aug 9, 2008
    Replies:
    5
    Views:
    886
    udayns
    Aug 11, 2008
  16. domainer_10

    HTTPS getting indexed instead of HTTP on wordpress

    domainer_10, May 26, 2008
    Replies:
    1
    Views:
    895
    domainer_10
    May 26, 2008
  17. Hostaff

    Excluding feeds in Wordpress

    Hostaff, May 22, 2007
    Replies:
    0
    Views:
    909
    Hostaff
    May 22, 2007
  18. surekhaweb

    robots.txt and delet the pages

    surekhaweb, Oct 2, 2007
    Replies:
    1
    Views:
    915
    Nucca_The_Warrior
    Oct 22, 2007
  19. Mr bloggy

    Will robots search even if I use web forwarding?

    Mr bloggy, Jan 9, 2008
    Replies:
    3
    Views:
    922
    SwapsRulez
    Jan 12, 2008
  20. tanha_dil

    Blocking particular subdomain urls.

    tanha_dil, Feb 27, 2010
    Replies:
    1
    Views:
    929
    strid3r
    Feb 27, 2010
  21. weilies

    Sitemap indexing for forum: Dynamic or Static is better?

    weilies, Sep 21, 2008
    Replies:
    0
    Views:
    933
    weilies
    Sep 21, 2008
  22. smartware

    What is User Agent?

    smartware, Apr 25, 2010
    Replies:
    2
    Views:
    937
    iloveindia
    May 3, 2010
  23. zinghana

    Robots.txt advice needed

    zinghana, Jun 18, 2010
    Replies:
    1
    Views:
    940
    manish.chauhan
    Jun 20, 2010
  24. carol123in

    robots.txt

    carol123in, Apr 26, 2010
    Replies:
    2
    Views:
    942
    carol123in
    May 3, 2010
  25. frio

    Excluding subdmains

    frio, Oct 30, 2007
    Replies:
    3
    Views:
    946
    Kuldeep1952
    Oct 30, 2007
  26. TheSites

    How to block out sessions from being indexed

    TheSites, Mar 29, 2008
    Replies:
    1
    Views:
    952
    manish.chauhan
    Apr 4, 2008
  27. alanbutt

    What are these dam Robots up to?

    alanbutt, Oct 31, 2008
    Replies:
    0
    Views:
    956
    alanbutt
    Oct 31, 2008
  28. catilucky

    robots.txt

    catilucky, Aug 2, 2010
    Replies:
    1
    Views:
    957
    zacharooni
    Aug 3, 2010
  29. clobberx

    Disallow php.ini

    clobberx, Jun 20, 2008
    Replies:
    1
    Views:
    959
    manish.chauhan
    Jun 21, 2008
  30. cheapbacklink

    site not indexing help needed

    cheapbacklink, Sep 30, 2008
    Replies:
    1
    Views:
    963
    keyaa
    Oct 3, 2008
  31. droog

    blocking duplicate content, wildcards

    droog, Aug 12, 2008
    Replies:
    1
    Views:
    963
    catanich
    Aug 21, 2008
  32. frogman49th

    How to use robots.txt in blogs?

    frogman49th, Oct 6, 2007
    Replies:
    0
    Views:
    966
    frogman49th
    Oct 6, 2007
  33. 1rapid

    help make a robots.txt

    1rapid, Jun 6, 2010
    Replies:
    3
    Views:
    966
    manish.chauhan
    Jun 11, 2010
  34. arcademan

    block root directory and all subdirectories?

    arcademan, Jun 29, 2010
    Replies:
    1
    Views:
    969
    manish.chauhan
    Jun 30, 2010
  35. needadwords

    Can Anyone Help...

    needadwords, Jul 5, 2008
    Replies:
    3
    Views:
    971
    catanich
    Aug 21, 2008
  36. monika007

    I wanna create robot.txt

    monika007, Jun 13, 2008
    Replies:
    2
    Views:
    973
    Dan Schulz
    Jun 13, 2008
  37. scuba5794

    should i change this and my robots.txt and how?

    scuba5794, Sep 6, 2007
    Replies:
    4
    Views:
    979
    seoworld
    Sep 11, 2007
  38. ruberr2002

    Please review my Robots.txt

    ruberr2002, Jun 23, 2008
    Replies:
    3
    Views:
    979
    manish.chauhan
    Jul 4, 2008
  39. Jimbob1208

    What is a Robots.txt

    Jimbob1208, Jan 10, 2009
    Replies:
    2
    Views:
    980
    manish.chauhan
    Jan 20, 2009
  40. Chalkywhite

    Blog logs and robots.txt

    Chalkywhite, Jul 8, 2010
    Replies:
    3
    Views:
    980
    manish.chauhan
    Jul 9, 2010
  41. nshadab

    Help required for ROBOTS.txt?

    nshadab, Jan 22, 2008
    Replies:
    4
    Views:
    981
    catanich
    Feb 2, 2008
  42. mahavir

    what all Robots.txt directives FAST supports ??

    mahavir, Dec 30, 2008
    Replies:
    2
    Views:
    981
    manish.chauhan
    Jan 20, 2009
  43. NaSh123

    Would this block?

    NaSh123, Aug 26, 2008
    Replies:
    2
    Views:
    983
    goliathus
    Aug 28, 2008
  44. kiransarv

    How to match patterns in robots.txt

    kiransarv, Nov 2, 2008
    Replies:
    1
    Views:
    983
    Aldo
    Nov 8, 2008
  45. ManchesterTech

    Robots.txt...what do I want from it?...

    ManchesterTech, May 20, 2008
    Replies:
    2
    Views:
    990
    manish.chauhan
    May 20, 2008
  46. cahyoaja

    robot.txt for blogspot

    cahyoaja, May 8, 2010
    Replies:
    1
    Views:
    991
    manish.chauhan
    Jun 11, 2010
  47. Honeygang

    Redirection help

    Honeygang, Dec 20, 2008
    Replies:
    3
    Views:
    995
    manish.chauhan
    Jan 20, 2009
  48. WorkFast

    What are the field values?

    WorkFast, Mar 18, 2009
    Replies:
    1
    Views:
    995
    manish.chauhan
    Mar 19, 2009
  49. internet money making

    Robot tag

    internet money making, Apr 29, 2010
    Replies:
    4
    Views:
    1,000
    lonew
    May 29, 2010
  50. uma

    IS there any tool to find out the particular page got spidered?

    uma, Sep 1, 2006
    Replies:
    0
    Views:
    1,005
    uma
    Sep 1, 2006
Showing threads 1 to 50 of 911

Thread Display Options

Loading...
Page 1 of 19