Google Not Crawling Some Url Structures on my site

Discussion in 'Search Engine Optimization' started by skysoftware, Aug 16, 2012.

  1. #1
    Hello Db Members.

    I Need Some Help From Someone Good in SEO to Fix A Problem I Faced With My New Site.

    Details:

    - The Website Consist of Two Main Url Structures :
    A - The first one contain one Parameter called (id) http://example.com/file.php?id=54d0c771b535bddd051c8539accbd493
    B- The Second one is a (Tag) or (Keyword) link http://example.com/p.php?q=lion+cache+cleaner

    - there is no problem with the second url structure (Tags Links) and i can see google bot crawl and index them.

    - Google bot even did not crawl the first url structure with (id) parameter and i can see google bot visit the main page then extracting all Tags links but he is never extract or visit the first type of url.

    - the first url structue that i have problem with it , its more rich contents from the tags links that indexed very well.

    - there is no problem in status code or header and the fetch as google bot option in wmt work great and return all internal links and 200 successfully status.

    ** I Hope if Someone Good in SEO Can Tell Me if the Hash Code in (id) parameter (32 long) (http://example.com/file.php?id=54d0c771b535bddd051c8539accbd493) Makes Any Problem with Google And Let him Ignore it and thank you in advance.
     
    skysoftware, Aug 16, 2012 IP
  2. ija61

    ija61 Member

    Messages:
    186
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    35
    #2
    Hmm...

    I do not have an explanation for this but, here is what it come in my mind...

    Did you submit a sitemap in WMT with those url?
    Do you get any error on the logs?
    I will suggest to use .htaccess to rewrite your first type of url's and build some social bookmarks on them....
    Add internal links form the second type of webpages to the first type...
     
    ija61, Aug 17, 2012 IP
  3. omegas

    omegas Peon

    Messages:
    21
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    me to have this type of problem that my home page have problem. google extracting the content and displays in serp not the meta tags.
     
    omegas, Aug 17, 2012 IP
  4. skysoftware

    skysoftware Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4

    thank you ija61 for your reply, yes when i notice this problem i submitted a sitemap and rss feed that contains the first type of links to WMT and there is no problem on them and i can see for example in wmt (20 links submitted and 0 links in google index) and this status stay as that for more than a month and non of them indexed.

    and yes there is internal links between the first type,second type and the home page and vise versa and i think google bot can easily find my links from multiple roots.

    Yes Sure I think you are right about rebuild the first type of links using .htaccess and using keyword on them but just i want to make sure if google bot has any problem with links that contains long mixed(chars + digits) parameter values (32 long) so i can find alternative way to rebuild them.
     
    skysoftware, Aug 17, 2012 IP
  5. ija61

    ija61 Member

    Messages:
    186
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    35
    #5
    I do not know what to say.... maybe a little search on the WMT guideline and help section can show us an answer but not sure.

    In theory they must be ok...

    One more thing that I do not remember... if look in yours log did you see any other boot crawling those type of url? It is possible and is just an opinion not a fact, that your url can be seen as a thread for sql injection or things like that and maybe this is the reason why G boots are not crawl them. But to be true I never hear something like this.

    I would be interested to share me what was the problem if you manage to find it.

    thank you
     
    ija61, Aug 17, 2012 IP
  6. skysoftware

    skysoftware Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6

    yes i can see yahoo & msn bots access these links successfully and many others bots, and i dont think any bots can see it as sql injection because thats parameter value only consists of digits and characters and there is no special characters on it.
     
    skysoftware, Aug 17, 2012 IP
  7. skysoftware

    skysoftware Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Hello,

    I'm Updating this post because i found the solution for my links problem and google bot start really crawl all my links and he is go deeper and deeper in internal links.

    just i make some changes to my url structure to fix this problem.

    old structure : example.com/file.php?id=54d0c771b535bddd051c8539accbd493
    new structure : example.com/file/54d0c771b535bddd051c8539accbd493/some-related-keywords

    Notice that i kept the hash code (54d0c771b535bddd051c8539accbd493) in both link types and it seems there is no problem google faced with that kinds of parameter values.

    the main change i made just to change url structure from scripting url to more friendly url and in fact i dont really know why google ignoring the first structure and he is start eating the second one so much like a hungry man (^_^).

    and i want to thanks ija61 for his useful replies on this post.

    have a nice day .. bye
     
    skysoftware, Aug 17, 2012 IP
  8. ajay pattnaik

    ajay pattnaik Peon

    Messages:
    79
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #8
    So according to you by using .htaccess we can get rid out of the problem. I agree with you. But what about WMT?
     
    ajay pattnaik, Aug 18, 2012 IP
  9. ija61

    ija61 Member

    Messages:
    186
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    35
    #9
    What is about webmaster tools? I can not understand your point? What WMT have to do with that?

    It was my pleasure... any time you need feel free to contact me.

    And one more question.. why is so important to keep that string in the url, how you use it? This if it is ok to tell me, if not here maybe in PM. But as I said if you can tell me if not it is ok for me.

    Thank you
     
    ija61, Aug 18, 2012 IP
  10. thosino

    thosino Member

    Messages:
    174
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    46
    #10
    I think there i some problem in your URL structuring. Please reconstruct your url structuring.
     
    thosino, Aug 19, 2012 IP
  11. macrosoft85

    macrosoft85 Member

    Messages:
    56
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    41
    #11
    You can use url rewriting for search engine friendly url.
     
    macrosoft85, Aug 20, 2012 IP
  12. ija61

    ija61 Member

    Messages:
    186
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    35
    #12
    You do realize that the OP have a post in witch he say that the problem was solved, you notice that right?
     
    ija61, Aug 20, 2012 IP
  13. Bestservices4u

    Bestservices4u Well-Known Member

    Messages:
    607
    Likes Received:
    12
    Best Answers:
    0
    Trophy Points:
    130
    #13
    Check Webmaster tool for any HTML errors.
     
    Bestservices4u, Sep 5, 2012 IP