Trying to exclude .php/ in robots.txt - how to?

Discussion in 'HTML & Website Design' started by roger_inkart, Nov 10, 2010.

  1. #1
    So, for some reason (probably a bad link somewhere) Google is insisting on looking for pages that are not there. Like so:

    http://www.inkart.net/art/wildlife_...jaguar/musk_ox/rocky_mountain_goat/sea_otter/

    But the problem is there are no directories after the .php in the site:

    http://www.inkart.net/art/wildlife_art/new_world_mammals.php

    I was told to remove the relative links and replace them with absolute. So, in the meantime I'd like to let Google know not to crawl anything ending in .php/

    Can I add the following to the robots.txt to discourage Google from trying to crawl non-existent links?

    Disallow: /*.php/

    Or will this tell Google to not crawl any of the .php files?

    Thanks for any feedback you could give.
     
    roger_inkart, Nov 10, 2010 IP