1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Twitter's robots.txt question:

Discussion in 'Search Engine Optimization' started by Hema1, Dec 9, 2010.

0
  1. #1
    Twitter's robots.txt, It shows everything is disallowed, but surprisingly search engines are crawling and indexing everybody's profiles pages, Why?
     
    Hema1, Dec 9, 2010 IP
  2. oilarts

    oilarts Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I have long assumed that the syntax of a robots.txt file was pretty straightforward, My question or problem is with the filespec part.
     
    oilarts, Dec 9, 2010 IP