Slurp Sees my files - w/o links??

Discussion in 'Yahoo' started by shina21, May 3, 2004.

  1. #1
    Hello,

    I am wondering how it is possible that slurp could see my links even though I have used mod_rewrite to make them very nice urls.

    For example, I am using wordpress blog software. I have checked all the links on all of the pages, to make sure of course the mod_rewrite rules worked. Somehow Slurp (and google) have actually spotted a blog posting site.com/index.ph?p=9 when there is really no way they could have found them!

    Obviously there is a way, or they wouldn't have found him. But what really sucks, is for some reason the post is now indexed under that URL!! The whole reason I have used to mod_rewrite rules is so that they wouldn't be - and my post title URLs would be instead! Arggggh...

    And I have checked the logs - the referer is "-" ... which stinks. I am not sure how they are finding these.

    Google is also finding the posts like this, but with a blank "-" referer also. However there is no leak, I promise! The blog is Here.

    I am very frustrated because I have no links, and for some reason it defies logic. Also, for some reason my page "site.com/" displays adsense ads that are different (and a PSA) then "site.com/index.php" - which are properly targeted ads. I am uncertain how fast the mediapartner googlebot comes to update, but on my other site it seems to happen very often.

    This is driving me nuts, because I don't want the wrong URLs indexed. The index.php?p=9.php one is on Google now too. Whacky...

    Finally, my friend and I submitted a total of 3 sites to yahoo with the free submission at the same time the other day. Two of our sites our indexed and have been visited a lot by slurp. The third hasn't even been visited once. That is screwy too - is it just the URL or how is slurp determining whether or not to visit? Is the site possibly blacklisted? We have not submitted it before. However, I am wondering if the submission went thru. I don't want to submit again and get blacklisted, but it is the one we want indexed most! =)

    Thanks in advance for any help!
     
    shina21, May 3, 2004 IP
  2. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,334
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #2
    Well, it has to have a link somewhere. My guess is someone offsite if linking to the p=9 page.

    - Shawn
     
    digitalpoint, May 3, 2004 IP
  3. shina21

    shina21 Peon

    Messages:
    3
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Thanks for the reply Shawn. Of course I have tried the link-path in google, where it is also indexed, and found nothing. I assume that is because anyone linking to it may not have a PR 4 or whatever. I thought I read that somewhere - something about a minimum pagerank for inbounds being recorded. Ah ... whacky. Also, I am not certain how anyone would be able to link to it if I used mod_rewrite to change the links ahead of time .... Also if someone were linking to it, wouldn't it show a referer in the logs? Instead it shows googlebot and "-" as the referer. Also, Googlebot is somehow seeing the other posts with no referer "?p=4" etc... I really have no clue but it doesnt add up. Perhaps I am simply a moron.

    Edit ** well i am still a moron, but i did find out that somehow the Geourl bot .. www.geourl.screwing with my brain ... is finding the posts as individuals. It is probably linking to me. So I will have to kill this bot, and make sure it knows whos the boss.

    Also, any ideas why #1 - Slurp chooses only 2 of 3 sites to visit immediately? Is it based on the URL or something (after submission on the yahoo add url) and #2 why in the world would adsense display targeted ads on www.site.com/index.php and different ads (a psa) on www.site.com/ (the more important one)?

    Thx so much!
     
    shina21, May 3, 2004 IP
  4. eCommando

    eCommando Well-Known Member

    Messages:
    115
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    103
    #4
    Link will show up as backlink on google if it's on the PR4+ page. Otherwise, it's there and counted, but won't show up on google backlink list.
     
    eCommando, May 14, 2004 IP
  5. dsr771

    dsr771 Peon

    Messages:
    65
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I have seen where google/msn have there own database of links the found in the past, and when I changed the page, they still looked at the old links. And since they still worked, I guess they will keep them.

    So, maybe google had previously seen that page before your mod-rewite.
     
    dsr771, May 16, 2004 IP
  6. nlopes

    nlopes Guest

    Messages:
    103
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    You can do a 301 redirect on those dynamic pages to the static ones.
     
    nlopes, May 17, 2004 IP
  7. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,334
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #7
    Even if you 301 a webpage, Yahoo still keeps the old one. I've had a website 301ed for almost a year, and Yahoo shows the old, not the new in the results. Lots of little things like that really makes me realize how much better Google is from a technical standpoint. I for one took Google for granted until I started seeing some technical limitations of Yahoo's bot.

    - Shawn
     
    digitalpoint, May 17, 2004 IP