Social Buttons
  1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 4 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. mariush

    Need advice on how to block certain page using robots.txt

    mariush, Aug 12, 2006
    Replies:
    7
    Views:
    10,025
    learn2success
    Jun 6, 2012
  2. hvwebseo

    How to stop crawling of multi language site

    hvwebseo, Jun 5, 2012
    Replies:
    1
    Views:
    2,342
    RoseHosting
    Jun 5, 2012
  3. matthecool

    Blocking .php from google in robots.txt

    matthecool, Jun 4, 2012
    Replies:
    1
    Views:
    2,418
    digitalpoint
    Jun 4, 2012
  4. AlisaRozzy

    Help regarding robots.txt file for my website

    AlisaRozzy, May 19, 2012
    Replies:
    1
    Views:
    2,414
    war_machine
    Jun 3, 2012
  5. riteshvyas52

    What is robots.txt ?

    riteshvyas52, May 4, 2012
    Replies:
    5
    Views:
    2,707
    JFSG
    Jun 3, 2012
  6. gdtechind

    How to Stop Web Robot from Engines other than G and Y and MSN

    gdtechind, Oct 27, 2005
    Replies:
    9
    Views:
    7,688
    Spearheadltd
    Jun 1, 2012
  7. joerudy1988

    robot.txt to allow everything

    joerudy1988, May 3, 2012
    Replies:
    3
    Views:
    3,051
    joerudy1988
    May 17, 2012
  8. satish nandanwar

    How we use robot.txt file for our site pages?

    satish nandanwar, May 9, 2012
    Replies:
    1
    Views:
    2,543
    bsdstr
    May 17, 2012
  9. TheSyndicate

    How do you get archiver to READ my robots?

    TheSyndicate, Mar 31, 2012
    Replies:
    6
    Views:
    2,892
    TheSyndicate
    Apr 27, 2012
  10. kshama

    I want to block folder /Portals/ and /portals/

    kshama, Apr 16, 2012
    Replies:
    1
    Views:
    2,604
    blacknet
    Apr 17, 2012
  11. sumitsaurabhmca

    robots.txt correct or not?

    sumitsaurabhmca, Apr 3, 2012
    Replies:
    7
    Views:
    2,590
    ryan_uk
    Apr 12, 2012
  12. Rajnish357

    What this robot.txt will do ? Please help !!

    Rajnish357, Jan 23, 2012
    Replies:
    2
    Views:
    3,071
    carlhenson
    Apr 11, 2012
  13. wijasenna

    does my robots.txt block my URLs?

    wijasenna, Apr 9, 2012
    Replies:
    3
    Views:
    3,213
    carlhenson
    Apr 11, 2012
  14. roynetwork7

    Noindex, nofollow on a wordpress blog

    roynetwork7, Mar 26, 2012
    Replies:
    2
    Views:
    2,935
    roynetwork7
    Apr 10, 2012
  15. sameerseo

    Robots.txt Allow/ Disallow Dynamic Pages

    sameerseo, Mar 26, 2012
    Replies:
    1
    Views:
    3,150
    tiffanywilliams12i2
    Apr 10, 2012
  16. matthecool

    blocking directory certain part of url from robots.txt

    matthecool, Oct 20, 2011
    Replies:
    3
    Views:
    3,924
    tiffanywilliams12i2
    Apr 10, 2012
  17. world pharmacare

    how to change the robots.txt file in open cart

    world pharmacare, Dec 31, 2011
    Replies:
    1
    Views:
    4,346
    tiffanywilliams12i2
    Apr 10, 2012
  18. godsgift

    robots.txt error when allowing all

    godsgift, Jan 9, 2012
    Replies:
    6
    Views:
    4,075
    tiffanywilliams12i2
    Apr 10, 2012
  19. TheSyndicate

    Robot directory

    TheSyndicate, Jan 24, 2012
    Replies:
    1
    Views:
    2,959
    tiffanywilliams12i2
    Apr 10, 2012
  20. qwikad.com

    Is there a way to make the bots respect the robots.txt file at all times?

    qwikad.com, Jan 31, 2012
    Replies:
    2
    Views:
    2,932
    tiffanywilliams12i2
    Apr 10, 2012
  21. aniket123

    Query about robots.txt function in wordpress?

    aniket123, Feb 1, 2012
    Replies:
    3
    Views:
    3,071
    tiffanywilliams12i2
    Apr 10, 2012
  22. varsha reddy

    yes it is important to keep robots.txt

    varsha reddy, Feb 1, 2012
    Replies:
    1
    Views:
    2,862
    tiffanywilliams12i2
    Apr 10, 2012
  23. rjd1265

    Robot.txt Kicking My A$$ Please Help!

    rjd1265, Feb 2, 2012
    Replies:
    2
    Views:
    2,624
    tiffanywilliams12i2
    Apr 10, 2012
  24. stilv

    how to get more yahoo and bing like google? [photo]

    stilv, Mar 3, 2012
    Replies:
    2
    Views:
    2,485
    tiffanywilliams12i2
    Apr 10, 2012
  25. AlisaRozzy

    Use of robot.txt file

    AlisaRozzy, Mar 23, 2012
    Replies:
    2
    Views:
    2,478
    tiffanywilliams12i2
    Apr 10, 2012
  26. arivin29

    thumbnail does not appear

    arivin29, Mar 26, 2012
    Replies:
    1
    Views:
    2,020
    tiffanywilliams12i2
    Apr 10, 2012
  27. TheSyndicate

    Is it normal that the /robots.txt is the most downloaded item from Server stats

    TheSyndicate, Mar 29, 2012
    Replies:
    11
    Views:
    2,431
    TheSyndicate
    Apr 9, 2012
  28. sofiaisabella01

    What is robot.txt flie?

    sofiaisabella01, Feb 19, 2012
    Replies:
    19
    Views:
    2,887
    mbitsol
    Apr 7, 2012
  29. NewComputer

    Question Re: Robots.txt

    NewComputer, Jun 12, 2004
    Replies:
    4
    Views:
    4,104
    anna30
    Apr 3, 2012
  30. mpchekuri

    How to block certain pages with url parameters from crawler's access using robots.txt

    mpchekuri, Feb 29, 2012
    Replies:
    4
    Views:
    2,652
    pindians007
    Mar 26, 2012
  31. abrahamgold

    Hello, I am new, and need some help on robots.txt

    abrahamgold, Oct 21, 2005
    Replies:
    14
    Views:
    8,157
    SPA assurance
    Mar 8, 2012
  32. sarahashlin

    How to create robots.txt file

    sarahashlin, Nov 28, 2011 ... 2
    Replies:
    22
    Views:
    6,524
    madison37
    Mar 6, 2012
  33. netboy

    Remove this stupid robots.txt

    netboy, Feb 22, 2008
    Replies:
    17
    Views:
    17,180
    Rituja
    Feb 15, 2012
  34. DeepakDwivedi

    Hi friends i want to disallow only this type of page in robot.txt

    DeepakDwivedi, Feb 9, 2012
    Replies:
    2
    Views:
    2,308
    DeepakDwivedi
    Feb 13, 2012
  35. rjd1265

    Robot.txt Troubles...again

    rjd1265, Feb 2, 2012
    Replies:
    0
    Views:
    1,785
    rjd1265
    Feb 2, 2012
  36. pla gilbert

    Webmaster Tools- Crawl errors Restricted by robots.txt ‎(14)‎

    pla gilbert, Jan 14, 2012
    Replies:
    2
    Views:
    2,692
    Cristina001
    Jan 26, 2012
  37. waleed.khaliqi

    google webmaster tool crawl error "Restricted by robots.txt ‎(473)‎"

    waleed.khaliqi, Jan 11, 2012
    Replies:
    1
    Views:
    2,214
    umerplz
    Jan 25, 2012
  38. ademmeda

    robots.txt and images

    ademmeda, Sep 11, 2011
    Replies:
    9
    Views:
    4,236
    WFgirls
    Jan 14, 2012
  39. iluvm

    Robot.txt file

    iluvm, Nov 30, 2005 ... 2
    Replies:
    20
    Views:
    11,809
    Emily Stephens
    Jan 3, 2012
  40. e-marketing guru

    Analyze These Robots.txt which one is better?

    e-marketing guru, Jun 23, 2008
    Replies:
    1
    Views:
    2,644
    TheSyndicate
    Dec 21, 2011
  41. TheSyndicate

    Amazon AWS Services - Cloudfront - FRA What to do with these?

    TheSyndicate, Dec 21, 2011
    Replies:
    0
    Views:
    2,146
    TheSyndicate
    Dec 21, 2011
  42. lok123

    Problem with timed out and unreacheable crawl error in google

    lok123, Dec 1, 2011
    Replies:
    2
    Views:
    2,464
    onlinegame6
    Dec 17, 2011
  43. wijasenna

    is this robot.txt good enough?

    wijasenna, Oct 25, 2011
    Replies:
    7
    Views:
    3,221
    xDiglett
    Dec 16, 2011
  44. mjindustry

    Quick Disallow Question for Gallery2 items

    mjindustry, Nov 24, 2011
    Replies:
    0
    Views:
    2,150
    mjindustry
    Nov 24, 2011
  45. TheSyndicate

    What is the best robot.txt for Wordpress a 2011

    TheSyndicate, Nov 15, 2011
    Replies:
    0
    Views:
    3,116
    TheSyndicate
    Nov 15, 2011
  46. aniket123

    Robots.txt Disallow: / function

    aniket123, Nov 1, 2011
    Replies:
    3
    Views:
    2,654
    tripbuilder
    Nov 14, 2011
  47. mbpalmer87

    What is everyone's favourite robot?

    mbpalmer87, Nov 9, 2011
    Replies:
    0
    Views:
    2,131
    mbpalmer87
    Nov 9, 2011
  48. MyArtGallery

    Please help me understanding robots.txt file

    MyArtGallery, Sep 6, 2011
    Replies:
    9
    Views:
    3,285
    pr0t0n
    Nov 8, 2011
  49. tradester

    robots.txt to prevent search engines from crawling/ indexing a page or folder

    tradester, Oct 13, 2011
    Replies:
    5
    Views:
    2,887
    bobwarner01
    Nov 4, 2011
  50. OSSEO

    How to restrict url from robots.txt file

    OSSEO, Sep 7, 2011
    Replies:
    11
    Views:
    3,525
    MyWebsiteNow
    Oct 17, 2011
Showing threads 151 to 200 of 909

Thread Display Options

Loading...
Page 4 of 19