robots.txt

Find help with setting up your robots.txt file on your web site.

Page 5 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. tradester

    robots.txt to prevent search engines from crawling/ indexing a page or folder

    tradester, Oct 13, 2011
    Replies:
    5
    Views:
    3,270
    bobwarner01
    Nov 4, 2011
  2. OSSEO

    How to restrict url from robots.txt file

    OSSEO, Sep 7, 2011
    Replies:
    11
    Views:
    4,146
    MyWebsiteNow
    Oct 17, 2011
  3. REtec

    Sub-domains and Robots.txt

    REtec, Sep 26, 2011
    Replies:
    2
    Views:
    3,117
    REtec
    Sep 29, 2011
  4. simplythewebst

    Some solution to prevent indexing my site except robots.txt

    simplythewebst, Sep 26, 2011
    Replies:
    1
    Views:
    2,694
    manig4
    Sep 27, 2011
  5. nikmahajan

    Blogger Robots.txt Problem- Google is not crawling

    nikmahajan, Aug 5, 2008
    Replies:
    8
    Views:
    10,636
    g4ngst3r
    Sep 25, 2011
  6. gogetit

    Robots.txt crawl error

    gogetit, Sep 20, 2011
    Replies:
    1
    Views:
    3,315
    aruninfo
    Sep 23, 2011
  7. Kalibaster

    Robots.txt for wordpress autoblog setup.

    Kalibaster, Sep 21, 2011
    Replies:
    0
    Views:
    3,292
    Kalibaster
    Sep 21, 2011
  8. www.worldofdth.in

    robot..txt file generator website

    www.worldofdth.in, Sep 6, 2011
    Replies:
    4
    Views:
    3,297
    yummysilent
    Sep 14, 2011
  9. VijayanKumar

    What Is Robots.txt?

    VijayanKumar, Jun 9, 2011 ... 2
    Replies:
    32
    Views:
    9,852
    webgfx
    Sep 14, 2011
  10. Rage

    how to create a page that search engine dont search???

    Rage, May 23, 2006
    Replies:
    5
    Views:
    4,319
    sophie1986
    Sep 5, 2011
  11. Gangakarthik

    Disable Feed pages

    Gangakarthik, Aug 30, 2011
    Replies:
    1
    Views:
    2,949
    babushkyn
    Sep 2, 2011
  12. Latestnewsin

    Urgently Google Rebot Help

    Latestnewsin, Aug 28, 2011
    Replies:
    1
    Views:
    2,784
    babushkyn
    Sep 2, 2011
  13. Ratman2050

    Is 32,000 Alexa Rank Good?

    Ratman2050, Oct 8, 2006 ... 2
    Replies:
    35
    Views:
    17,394
    icyfiren
    Aug 26, 2011
  14. Arcos

    How to STOP indexing web pages

    Arcos, Dec 23, 2005
    Replies:
    14
    Views:
    8,562
    jhon786
    Aug 25, 2011
  15. bluestar

    How to set right crawler errors?

    bluestar, Aug 1, 2011
    Replies:
    8
    Views:
    4,117
    jhon786
    Aug 25, 2011
  16. LanceT

    Will robots.txt remove my crawl errors

    LanceT, Aug 9, 2011
    Replies:
    1
    Views:
    3,479
    Chuman
    Aug 14, 2011
  17. Planet_x

    Block Querystring for all user-agents

    Planet_x, Aug 2, 2011
    Replies:
    1
    Views:
    3,620
    manish.chauhan
    Aug 5, 2011
  18. U-Tag

    My blog "robots.txt" contain any indexing error?

    U-Tag, Aug 4, 2011
    Replies:
    2
    Views:
    2,996
    U-Tag
    Aug 5, 2011
  19. xenoo

    Using Robots.txt for image hosting site?

    xenoo, Jun 24, 2011
    Replies:
    1
    Views:
    3,209
    manish.chauhan
    Aug 1, 2011
  20. Fking

    The latest posts page in WP brings duplication issues.

    Fking, Jul 8, 2011
    Replies:
    1
    Views:
    2,895
    manish.chauhan
    Aug 1, 2011
  21. jducey1234

    Using robots.txt for better search engine placement

    jducey1234, Jun 1, 2011
    Replies:
    10
    Views:
    4,187
    manish.chauhan
    Aug 1, 2011
  22. electroze

    Help! Yahoo crawler is sucking my bandwidth - 8,779 queries in last 3 days!

    electroze, Jul 20, 2011
    Replies:
    1
    Views:
    3,009
    manish.chauhan
    Aug 1, 2011
  23. i_am_dhaval

    is am i right with robots.txt file

    i_am_dhaval, Jul 11, 2011
    Replies:
    6
    Views:
    3,234
    manish.chauhan
    Aug 1, 2011
  24. NickMP3

    block all sub domains

    NickMP3, Jul 10, 2011
    Replies:
    3
    Views:
    3,769
    manish.chauhan
    Aug 1, 2011
  25. aj845

    unknown bots.

    aj845, Jul 6, 2007
    Replies:
    12
    Views:
    11,499
    cquinndesign
    Jul 3, 2011
  26. mjindustry

    Is this going to work? (robots.txt)

    mjindustry, May 3, 2011
    Replies:
    8
    Views:
    3,879
    upendraets
    Jun 25, 2011
  27. shaun_ru_th

    correct 301 redirect methord???

    shaun_ru_th, May 31, 2011
    Replies:
    5
    Views:
    3,723
    upendraets
    Jun 25, 2011
  28. micmic

    Flash files in robots.txt

    micmic, Apr 27, 2007
    Replies:
    5
    Views:
    6,431
    jbtooloo
    Jun 22, 2011
  29. upendraets

    how we make Robot.txt and how we add in our site?

    upendraets, Jun 17, 2011
    Replies:
    1
    Views:
    3,076
    seodash
    Jun 17, 2011
  30. sandeeppandey

    Robot.Txt And RSS

    sandeeppandey, Jun 2, 2011
    Replies:
    4
    Views:
    987
    WishBone
    Jun 7, 2011
  31. prclicks

    Create robots.txt File, Need Your Suggestion

    prclicks, Jun 3, 2011
    Replies:
    0
    Views:
    2,761
    prclicks
    Jun 3, 2011
  32. AlwaysThinking

    How to stop Gugl from indexing https urls as duplicates in wordpress since adding SSL

    AlwaysThinking, May 9, 2011
    Replies:
    4
    Views:
    3,770
    thejimy
    May 29, 2011
  33. premierfabio

    Robots Problem

    premierfabio, May 23, 2011
    Replies:
    6
    Views:
    3,237
    RamCity
    May 24, 2011
  34. mlao

    Disallow Sub-domain From Robots.txt ???

    mlao, May 7, 2011
    Replies:
    8
    Views:
    6,451
    manish.chauhan
    May 24, 2011
  35. egreenr2

    robots.txtx support

    egreenr2, May 12, 2011
    Replies:
    1
    Views:
    2,608
    manish.chauhan
    May 18, 2011
  36. guzelonline

    restrict co.cc hack with robots.txt

    guzelonline, May 9, 2011
    Replies:
    1
    Views:
    3,196
    manish.chauhan
    May 18, 2011
  37. vnpatra2

    robots.txt help

    vnpatra2, May 6, 2011
    Replies:
    4
    Views:
    2,823
    manish.chauhan
    May 18, 2011
  38. frustik

    what disallow or allow in a wp blog

    frustik, May 17, 2011
    Replies:
    1
    Views:
    2,521
    manish.chauhan
    May 18, 2011
  39. style fashion

    Best Robots.txt For Wordpress Blog ....

    style fashion, Oct 1, 2009
    Replies:
    13
    Views:
    10,904
    mohisha
    May 10, 2011
  40. covetousrat

    Can someone check my Robot.txt My site is still not indexed yet.

    covetousrat, Mar 27, 2011
    Replies:
    11
    Views:
    4,413
    us2006
    May 5, 2011
  41. dkv

    Some posts Restricted by robots.txt in Google webmaster tool

    dkv, Mar 5, 2011
    Replies:
    3
    Views:
    3,230
    manish.chauhan
    May 4, 2011
  42. shakac

    google maybe banneding my account Help me in robots.txt file

    shakac, Mar 9, 2011
    Replies:
    2
    Views:
    2,888
    manish.chauhan
    May 4, 2011
  43. besbesmany

    please help problem blogger restricted all my posts

    besbesmany, Feb 24, 2011
    Replies:
    2
    Views:
    2,906
    manish.chauhan
    May 4, 2011
  44. rajeev_seo

    Robots.tst generate

    rajeev_seo, Apr 17, 2011
    Replies:
    1
    Views:
    2,503
    manish.chauhan
    May 4, 2011
  45. harryoxford

    Robots.txt

    harryoxford, Feb 10, 2011
    Replies:
    14
    Views:
    3,893
    manish.chauhan
    May 4, 2011
  46. allanswagger

    Robots.txt

    allanswagger, Sep 2, 2010
    Replies:
    17
    Views:
    4,382
    manish.chauhan
    May 4, 2011
  47. clive2

    robots.txt explanation need?

    clive2, Apr 25, 2011
    Replies:
    2
    Views:
    2,953
    manish.chauhan
    May 4, 2011
  48. tb1234

    unknown bots crawling? how are they and what do they want?

    tb1234, Jun 7, 2006
    Replies:
    4
    Views:
    3,663
    manish.chauhan
    May 3, 2011
  49. gagan.kr90

    What is Robot.txt File?

    gagan.kr90, Apr 25, 2011
    Replies:
    3
    Views:
    2,449
    manish.chauhan
    May 3, 2011
  50. sarahturned

    Help Understanding Robots.txt basic commands

    sarahturned, Dec 30, 2010
    Replies:
    5
    Views:
    3,392
    brad.smith4321
    Apr 19, 2011
Showing threads 201 to 250 of 911

Thread Display Options

Loading...
Page 5 of 19