How To Crawl An Ajax Url

Discussion in 'Google' started by jorgepson, Feb 13, 2013.

  1. #1
    first i want to say sorry, i don't know where i should place this post.

    we are facing a problem with ajax. the url generated from ajax which has the hash tag(#) are not been indexed by google. i know that google doesn't have the capability to crawl ajax so my question is if i create a sitemap pointing to the product pages which has # so could it be possible that google will now crawl that pages? or there's an element that i should use in order for the bots to penetrate the url with hash tags?

    thanks.
     
    jorgepson, Feb 13, 2013 IP
  2. ChrisRey

    ChrisRey Greenhorn

    Messages:
    168
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    18
    #2
    Modify your server so that it processes URLs with "?_escaped_fragment_=" in them to map to an HTML snapshot of the corresponding "#!" Web page. An HTML snapshot contains all the Web page's content that shows up after any JavaScript is done executing. A headless browser such as HtmlUnit can retrieve HTML snapshots for you. Make sure the URL is unescaped with standard URL decoding.
     
    ChrisRey, Feb 13, 2013 IP
  3. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,334
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #3
    Is there a reason your app needs to be AJAX driven? At least for the content you want spiderable?

    Google can crawl AJAX content (which obviously you know because you know about the hash tag in the URL), but it's probably not going to be as good as crawling non-AJAX pages. I suspect this is why Twitter was using AJAX URLs with hashtags for awhile, but then switched back to more traditional ones.

    For those that *don't* know, but need the info, here's some reading:

    https://developers.google.com/webmasters/ajax-crawling/docs/specification
     
    digitalpoint, Feb 13, 2013 IP
  4. jorgepson

    jorgepson Active Member

    Messages:
    103
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    73
    #4
    thanks for the response. well i am studying this https://developers.google.com/webmasters/ajax-crawling/docs/specification. hope i can get it right.
     
    jorgepson, Feb 13, 2013 IP