1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Can I add specific urls to robots.txt to stop google indexing them?

Discussion in 'robots.txt' started by David Lowe, Jun 25, 2013.

0
  1. #1
    I have a website that lists hundreds of events (football matches, car boot sales, tribute nights etc). When the events expire is there a way using the robots.txt that they can automatically be hidden from search engines?
     
    David Lowe, Jun 25, 2013 IP
  2. Helge Sverre

    Helge Sverre Prominent Member Affiliate Manager

    Messages:
    840
    Likes Received:
    99
    Best Answers:
    2
    Trophy Points:
    305
    Digital Goods:
    2
    #2
    I'm assuming you are using a database to track the event status
    I also assume that when the event is expired you change a value in the database(for example active = false)
    Pseudo-code for this:

    if event is not active
    write new line to robots.txt with text "Disallow: /event/id/"

    Or something like that, if you ask a php developer he could probably whip you up a script like that in an hour or less.
     
    Helge Sverre, Jun 28, 2013 IP
  3. BabyDoll123

    BabyDoll123 Banned

    Messages:
    15
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    21
    #3
    yes sure you can add specific URLs to stop Google indexing them
    Use this string >
    Disallow: / URL
     
    BabyDoll123, Jun 19, 2014 IP