1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 14 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. nino2007

    What is the problem with Google and my site?

    nino2007, Jan 22, 2008
    Replies:
    3
    Views:
    1,528
    catanich
    Feb 2, 2008
  2. kong707

    is robot.txt very important for a forum?

    kong707, Dec 18, 2007
    Replies:
    8
    Views:
    1,898
    Chris-777
    Feb 2, 2008
  3. dalailama

    avoid robots to index mainsite.com/undersite.com

    dalailama, Jan 11, 2008
    Replies:
    3
    Views:
    1,685
    rich_uk
    Jan 23, 2008
  4. login

    Avoid indexing of ../lyrics/print,74306.html

    login, Jan 11, 2008
    Replies:
    14
    Views:
    1,342
    SwapsRulez
    Jan 14, 2008
  5. crystak

    Difference between /category/ and /category disallowing ?

    crystak, Dec 29, 2007
    Replies:
    3
    Views:
    1,051
    SwapsRulez
    Jan 12, 2008
  6. ecctao

    Is it necessary to write a list of my all my pages in a tobots.txt

    ecctao, Jan 1, 2008
    Replies:
    4
    Views:
    1,107
    SwapsRulez
    Jan 12, 2008
  7. lowprofilejoe

    Disallow Flash Files

    lowprofilejoe, Jan 4, 2008
    Replies:
    7
    Views:
    1,877
    SwapsRulez
    Jan 12, 2008
  8. Mr bloggy

    Will robots search even if I use web forwarding?

    Mr bloggy, Jan 9, 2008
    Replies:
    3
    Views:
    922
    SwapsRulez
    Jan 12, 2008
  9. johan-cr

    Is Dreamhost support stupid or am i missing something?

    johan-cr, Dec 27, 2006 ... 2
    Replies:
    33
    Views:
    16,092
    danieloffice
    Jan 9, 2008
  10. Browning

    I need a robots.txt for my proxy site. Don't know what goes in/how to do this

    Browning, Dec 25, 2007
    Replies:
    2
    Views:
    1,339
    lpstong
    Dec 26, 2007
  11. gree124

    Exert rebots.txt

    gree124, Dec 3, 2007
    Replies:
    2
    Views:
    1,347
    just-4-teens
    Dec 23, 2007
  12. eathosting.com

    What is a Robot???

    eathosting.com, Dec 1, 2007
    Replies:
    7
    Views:
    1,387
    gab1982
    Dec 20, 2007
  13. Mysmasken

    robots.txt to deny all spiders for proxy?

    Mysmasken, Nov 29, 2007
    Replies:
    3
    Views:
    20,064
    Mysmasken
    Nov 30, 2007
  14. MrThePlague

    Follow signup.php but NOT the proceeding query string

    MrThePlague, Nov 27, 2007
    Replies:
    1
    Views:
    1,475
    Kuldeep1952
    Nov 28, 2007
  15. acrimon

    Sort of a catch-22 here.

    acrimon, Nov 15, 2007
    Replies:
    4
    Views:
    1,251
    Ladadadada
    Nov 22, 2007
  16. Avalanche

    Robots.txt for Virtual Subdomains?

    Avalanche, Nov 14, 2007
    Replies:
    1
    Views:
    1,182
    energy.fs
    Nov 21, 2007
  17. Pammer

    How to block keywords via robots.txt ?

    Pammer, Nov 19, 2007
    Replies:
    2
    Views:
    3,218
    Pammer
    Nov 19, 2007
  18. Seobiznezz

    What should a robots.txt say?

    Seobiznezz, Nov 7, 2007
    Replies:
    4
    Views:
    1,310
    silence
    Nov 17, 2007
  19. schwab

    robots.txt

    schwab, Nov 12, 2007
    Replies:
    5
    Views:
    1,102
    schwab
    Nov 13, 2007
  20. snooks

    Robots .txt restricted - assistance needed

    snooks, Nov 9, 2007
    Replies:
    2
    Views:
    1,667
    snooks
    Nov 9, 2007
  21. ||NeX||

    Let index just the HP

    ||NeX||, Nov 3, 2007
    Replies:
    4
    Views:
    1,039
    ||NeX||
    Nov 9, 2007
  22. sirisaacnewton2007

    Can I tell spiders to only come at night?

    sirisaacnewton2007, Oct 17, 2007
    Replies:
    8
    Views:
    1,234
    harrysmith
    Nov 7, 2007
  23. Armaan143

    How to make robots.txt file for my site

    Armaan143, Oct 24, 2007
    Replies:
    8
    Views:
    5,118
    Armaan143
    Nov 6, 2007
  24. softarea51

    Limit bot access frequency

    softarea51, Nov 1, 2007
    Replies:
    7
    Views:
    5,228
    ajsa52
    Nov 5, 2007
  25. rayqsl

    Do I Need A Robots.txt?

    rayqsl, Oct 13, 2007
    Replies:
    11
    Views:
    1,594
    Kuldeep1952
    Nov 2, 2007
  26. login

    block pages in robots.txt

    login, Nov 2, 2007
    Replies:
    2
    Views:
    2,232
    Monty
    Nov 2, 2007
  27. Boardwalk

    Robots.txt Guide

    Boardwalk, Oct 16, 2007
    Replies:
    6
    Views:
    1,847
    hans
    Nov 1, 2007
  28. compindustries

    Help with robots.txt file

    compindustries, Jun 19, 2006
    Replies:
    13
    Views:
    2,846
    muks
    Nov 1, 2007
  29. frio

    Excluding subdmains

    frio, Oct 30, 2007
    Replies:
    3
    Views:
    946
    Kuldeep1952
    Oct 30, 2007
  30. clickalot

    http code 200 mystery

    clickalot, Oct 13, 2007
    Replies:
    4
    Views:
    1,763
    clickalot
    Oct 23, 2007
  31. surekhaweb

    robots.txt and delet the pages

    surekhaweb, Oct 2, 2007
    Replies:
    1
    Views:
    915
    Nucca_The_Warrior
    Oct 22, 2007
  32. bad_bob00

    Error pages on my site - robots.txt ?

    bad_bob00, Sep 21, 2007
    Replies:
    7
    Views:
    1,492
    Nucca_The_Warrior
    Oct 22, 2007
  33. EducationLinks

    How to block archive.org bot with both robots.txt AND .htaccess

    EducationLinks, Aug 9, 2007
    Replies:
    6
    Views:
    14,663
    hans
    Oct 21, 2007
  34. greatlogix

    Adding 301 redirect for more than 1000 pages

    greatlogix, Oct 3, 2007
    Replies:
    1
    Views:
    1,582
    Ladadadada
    Oct 14, 2007
  35. n0chance

    301 Redirect the robots.txt outside the root domain?

    n0chance, Oct 9, 2007
    Replies:
    3
    Views:
    3,492
    mcfox
    Oct 9, 2007
  36. frogman49th

    How to use robots.txt in blogs?

    frogman49th, Oct 6, 2007
    Replies:
    0
    Views:
    966
    frogman49th
    Oct 6, 2007
  37. Markas

    Does robots.txt work for all search engines?

    Markas, Sep 29, 2007
    Replies:
    5
    Views:
    2,909
    harrysmith
    Oct 2, 2007
  38. birdsq

    Only googlebot,slurp and msnbot should crawl my site.

    birdsq, Sep 10, 2007
    Replies:
    5
    Views:
    2,719
    harrysmith
    Sep 28, 2007
  39. blabla11

    Name of yahoo, google and msn bot

    blabla11, Aug 8, 2007
    Replies:
    9
    Views:
    19,250
    seoworld
    Sep 11, 2007
  40. scuba5794

    should i change this and my robots.txt and how?

    scuba5794, Sep 6, 2007
    Replies:
    4
    Views:
    979
    seoworld
    Sep 11, 2007
  41. Abhik

    How can I block a particular section of my forum?

    Abhik, Aug 29, 2007
    Replies:
    1
    Views:
    1,059
    evera
    Sep 10, 2007
  42. ThomaSeidler

    Help in encouraging Yahoo and MSN robots to obedience

    ThomaSeidler, Jun 14, 2007
    Replies:
    12
    Views:
    2,557
    evera
    Aug 28, 2007
  43. Diablos

    Standard Disallowed Pages with Robots.txt

    Diablos, Aug 23, 2007
    Replies:
    2
    Views:
    1,125
    Directory_SEO
    Aug 27, 2007
  44. sufi

    How to block images directory?

    sufi, May 26, 2007
    Replies:
    3
    Views:
    2,775
    trichnosis
    Aug 27, 2007
  45. cocaine

    I dont know the very basic thing about robots.txt

    cocaine, Feb 26, 2007
    Replies:
    10
    Views:
    2,141
    twistedspikes
    Aug 8, 2007
  46. st_jimi

    Can someone check my robots.txt file

    st_jimi, Jul 2, 2007
    Replies:
    3
    Views:
    1,625
    trichnosis
    Aug 7, 2007
  47. turboadds

    Traffic bot

    turboadds, May 7, 2007
    Replies:
    2
    Views:
    1,625
    trichnosis
    Aug 7, 2007
  48. danieloffice

    A default robots.txt for all my site.......... Please review

    danieloffice, Jul 13, 2007
    Replies:
    1
    Views:
    1,412
    trichnosis
    Aug 7, 2007
  49. ArtfulWebSites

    Bad Bot Advice

    ArtfulWebSites, Mar 23, 2007
    Replies:
    10
    Views:
    2,663
    trichnosis
    Aug 6, 2007
  50. Audiomad

    how many sitemaps do I put in the robots.txt file

    Audiomad, Jul 10, 2007
    Replies:
    2
    Views:
    1,568
    trichnosis
    Aug 6, 2007
Showing threads 651 to 700 of 911

Thread Display Options

Loading...
Page 14 of 19