robots.txt

Find help with setting up your robots.txt file on your web site.

Page 15 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. Gangakarthik

    Disable Feed pages

    Gangakarthik, Aug 30, 2011
    Replies:
    1
    Views:
    2,939
    babushkyn
    Sep 2, 2011
  2. Rage

    how to create a page that search engine dont search???

    Rage, May 23, 2006
    Replies:
    5
    Views:
    4,311
    sophie1986
    Sep 5, 2011
  3. VijayanKumar

    What Is Robots.txt?

    VijayanKumar, Jun 9, 2011 ... 2
    Replies:
    32
    Views:
    9,821
    webgfx
    Sep 14, 2011
  4. www.worldofdth.in

    robot..txt file generator website

    www.worldofdth.in, Sep 6, 2011
    Replies:
    4
    Views:
    3,287
    yummysilent
    Sep 14, 2011
  5. Kalibaster

    Robots.txt for wordpress autoblog setup.

    Kalibaster, Sep 21, 2011
    Replies:
    0
    Views:
    3,286
    Kalibaster
    Sep 21, 2011
  6. gogetit

    Robots.txt crawl error

    gogetit, Sep 20, 2011
    Replies:
    1
    Views:
    3,308
    aruninfo
    Sep 23, 2011
  7. nikmahajan

    Blogger Robots.txt Problem- Google is not crawling

    nikmahajan, Aug 5, 2008
    Replies:
    8
    Views:
    10,622
    g4ngst3r
    Sep 25, 2011
  8. simplythewebst

    Some solution to prevent indexing my site except robots.txt

    simplythewebst, Sep 26, 2011
    Replies:
    1
    Views:
    2,689
    manig4
    Sep 27, 2011
  9. REtec

    Sub-domains and Robots.txt

    REtec, Sep 26, 2011
    Replies:
    2
    Views:
    3,107
    REtec
    Sep 29, 2011
  10. OSSEO

    How to restrict url from robots.txt file

    OSSEO, Sep 7, 2011
    Replies:
    11
    Views:
    4,135
    MyWebsiteNow
    Oct 17, 2011
  11. tradester

    robots.txt to prevent search engines from crawling/ indexing a page or folder

    tradester, Oct 13, 2011
    Replies:
    5
    Views:
    3,264
    bobwarner01
    Nov 4, 2011
  12. MyArtGallery

    Please help me understanding robots.txt file

    MyArtGallery, Sep 6, 2011
    Replies:
    9
    Views:
    3,911
    pr0t0n
    Nov 8, 2011
  13. mbpalmer87

    What is everyone's favourite robot?

    mbpalmer87, Nov 9, 2011
    Replies:
    0
    Views:
    2,420
    mbpalmer87
    Nov 9, 2011
  14. aniket123

    Robots.txt Disallow: / function

    aniket123, Nov 1, 2011
    Replies:
    3
    Views:
    3,050
    tripbuilder
    Nov 14, 2011
  15. TheSyndicate

    What is the best robot.txt for Wordpress a 2011

    TheSyndicate, Nov 15, 2011
    Replies:
    0
    Views:
    3,430
    TheSyndicate
    Nov 15, 2011
  16. mjindustry

    Quick Disallow Question for Gallery2 items

    mjindustry, Nov 24, 2011
    Replies:
    0
    Views:
    2,490
    mjindustry
    Nov 24, 2011
  17. wijasenna

    is this robot.txt good enough?

    wijasenna, Oct 25, 2011
    Replies:
    7
    Views:
    3,731
    xDiglett
    Dec 16, 2011
  18. lok123

    Problem with timed out and unreacheable crawl error in google

    lok123, Dec 1, 2011
    Replies:
    2
    Views:
    2,808
    onlinegame6
    Dec 17, 2011
  19. TheSyndicate

    Amazon AWS Services - Cloudfront - FRA What to do with these?

    TheSyndicate, Dec 21, 2011
    Replies:
    0
    Views:
    2,951
    TheSyndicate
    Dec 21, 2011
  20. e-marketing guru

    Analyze These Robots.txt which one is better?

    e-marketing guru, Jun 23, 2008
    Replies:
    1
    Views:
    2,942
    TheSyndicate
    Dec 21, 2011
  21. iluvm

    Robot.txt file

    iluvm, Nov 30, 2005 ... 2
    Replies:
    20
    Views:
    12,727
    Emily Stephens
    Jan 3, 2012
  22. ademmeda

    robots.txt and images

    ademmeda, Sep 11, 2011
    Replies:
    9
    Views:
    4,764
    WFgirls
    Jan 14, 2012
  23. waleed.khaliqi

    google webmaster tool crawl error "Restricted by robots.txt ‎(473)‎"

    waleed.khaliqi, Jan 11, 2012
    Replies:
    1
    Views:
    2,554
    umerplz
    Jan 25, 2012
  24. pla gilbert

    Webmaster Tools- Crawl errors Restricted by robots.txt ‎(14)‎

    pla gilbert, Jan 14, 2012
    Replies:
    2
    Views:
    3,026
    Cristina001
    Jan 26, 2012
  25. rjd1265

    Robot.txt Troubles...again

    rjd1265, Feb 2, 2012
    Replies:
    0
    Views:
    2,053
    rjd1265
    Feb 2, 2012
  26. DeepakDwivedi

    Hi friends i want to disallow only this type of page in robot.txt

    DeepakDwivedi, Feb 9, 2012
    Replies:
    2
    Views:
    2,590
    DeepakDwivedi
    Feb 13, 2012
  27. netboy

    Remove this stupid robots.txt

    netboy, Feb 22, 2008
    Replies:
    17
    Views:
    17,898
    Rituja
    Feb 15, 2012
  28. sarahashlin

    How to create robots.txt file

    sarahashlin, Nov 28, 2011 ... 2
    Replies:
    22
    Views:
    7,615
    madison37
    Mar 6, 2012
  29. abrahamgold

    Hello, I am new, and need some help on robots.txt

    abrahamgold, Oct 21, 2005
    Replies:
    14
    Views:
    9,237
    SPA assurance
    Mar 8, 2012
  30. mpchekuri

    How to block certain pages with url parameters from crawler's access using robots.txt

    mpchekuri, Feb 29, 2012
    Replies:
    4
    Views:
    3,238
    pindians007
    Mar 26, 2012
  31. NewComputer

    Question Re: Robots.txt

    NewComputer, Jun 12, 2004
    Replies:
    4
    Views:
    4,446
    anna30
    Apr 3, 2012
  32. sofiaisabella01

    What is robot.txt flie?

    sofiaisabella01, Feb 19, 2012
    Replies:
    19
    Views:
    3,523
    mbitsol
    Apr 7, 2012
  33. TheSyndicate

    Is it normal that the /robots.txt is the most downloaded item from Server stats

    TheSyndicate, Mar 29, 2012
    Replies:
    11
    Views:
    2,792
    TheSyndicate
    Apr 9, 2012
  34. arivin29

    thumbnail does not appear

    arivin29, Mar 26, 2012
    Replies:
    1
    Views:
    2,307
    tiffanywilliams12i2
    Apr 10, 2012
  35. AlisaRozzy

    Use of robot.txt file

    AlisaRozzy, Mar 23, 2012
    Replies:
    2
    Views:
    3,101
    tiffanywilliams12i2
    Apr 10, 2012
  36. stilv

    how to get more yahoo and bing like google? [photo]

    stilv, Mar 3, 2012
    Replies:
    2
    Views:
    2,781
    tiffanywilliams12i2
    Apr 10, 2012
  37. rjd1265

    Robot.txt Kicking My A$$ Please Help!

    rjd1265, Feb 2, 2012
    Replies:
    2
    Views:
    2,955
    tiffanywilliams12i2
    Apr 10, 2012
  38. varsha reddy

    yes it is important to keep robots.txt

    varsha reddy, Feb 1, 2012
    Replies:
    1
    Views:
    3,184
    tiffanywilliams12i2
    Apr 10, 2012
  39. aniket123

    Query about robots.txt function in wordpress?

    aniket123, Feb 1, 2012
    Replies:
    3
    Views:
    3,523
    tiffanywilliams12i2
    Apr 10, 2012
  40. qwikad.com

    Is there a way to make the bots respect the robots.txt file at all times?

    qwikad.com, Jan 31, 2012
    Replies:
    2
    Views:
    3,240
    tiffanywilliams12i2
    Apr 10, 2012
  41. TheSyndicate

    Robot directory

    TheSyndicate, Jan 24, 2012
    Replies:
    1
    Views:
    3,245
    tiffanywilliams12i2
    Apr 10, 2012
  42. godsgift

    robots.txt error when allowing all

    godsgift, Jan 9, 2012
    Replies:
    6
    Views:
    4,623
    tiffanywilliams12i2
    Apr 10, 2012
  43. world pharmacare

    how to change the robots.txt file in open cart

    world pharmacare, Dec 31, 2011
    Replies:
    1
    Views:
    4,775
    tiffanywilliams12i2
    Apr 10, 2012
  44. matthecool

    blocking directory certain part of url from robots.txt

    matthecool, Oct 20, 2011
    Replies:
    3
    Views:
    4,359
    tiffanywilliams12i2
    Apr 10, 2012
  45. sameerseo

    Robots.txt Allow/ Disallow Dynamic Pages

    sameerseo, Mar 26, 2012
    Replies:
    1
    Views:
    3,613
    tiffanywilliams12i2
    Apr 10, 2012
  46. roynetwork7

    Noindex, nofollow on a wordpress blog

    roynetwork7, Mar 26, 2012
    Replies:
    2
    Views:
    3,287
    roynetwork7
    Apr 10, 2012
  47. wijasenna

    does my robots.txt block my URLs?

    wijasenna, Apr 9, 2012
    Replies:
    3
    Views:
    3,703
    carlhenson
    Apr 11, 2012
  48. Rajnish357

    What this robot.txt will do ? Please help !!

    Rajnish357, Jan 23, 2012
    Replies:
    2
    Views:
    3,516
    carlhenson
    Apr 11, 2012
  49. sumitsaurabhmca

    robots.txt correct or not?

    sumitsaurabhmca, Apr 3, 2012
    Replies:
    7
    Views:
    3,100
    ryan_uk
    Apr 12, 2012
  50. kshama

    I want to block folder /Portals/ and /portals/

    kshama, Apr 16, 2012
    Replies:
    1
    Views:
    3,015
    blacknet
    Apr 17, 2012
Showing threads 701 to 750 of 911

Thread Display Options

Loading...
Page 15 of 19