Problems with Slurp and Atom

Discussion in 'Yahoo' started by nevetS, Jan 9, 2007.

  1. #1
    I've been getting a lot of 404's in my logs and I decided to track them down.

    
    72.30.226.208 - - [09/Jan/2007:11:30:20 -0800] "GET /blog/flinstones_for_winstons.htmlhttp://www.stevekallestad.com/mov/mt-atom.cgi/weblog/blog_id=1/entry_id=85 HTTP/1.0" 404 249 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
    
    Code (markup):
    I couldn't figure out why a spider would be combining url's like that. Then I noticed something about the url - the second part of the combined url's referenced mt-atom.cgi - and I know (at least I thought I knew) that there are no links to mt-atom.cgi on my site.

    I grepped for mt-atom.cgi and I found references in my auto-generated atom.xml feed from movable type. Apparently Movable Type has an atom api, and these links are meant to be consumed by applications that consume the atom api. They look like this:
    
    <link rel="service.edit" type="application/atom+xml" href="http://www.stevekallestad.com/mov/mt-atom.cgi/weblog/blog_id=1/entry_id=137" title="Pipelining and Task Management" />
    
    Code (markup):
    Apparently, the Slurp bot was combining the main url with the service.edit url referenced in atom.xml. There's no reason to do something like this - it is a bug on Yahoo's side. Since I don't actually use the atom api, I removed the references from the template.

    If you run Movable Type, and you see Slurp doing what it was doing to my site, you probably want to get rid of them too - or get rid of the atom.xml feed all together.
     
    nevetS, Jan 9, 2007 IP