googlebot stuck?

Discussion in 'Google' started by samantha pia, Dec 21, 2004.

  1. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #41
    But that's not good -- you should redirect those to either a custom 404 page or to your index page.
     
    minstrel, Jan 10, 2005 IP
  2. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #42
    i have a custom 404 page with the sites logo and menu, how would i send them to it?

    i get lots like that, 1600+ in 1 day :(
    whats the code to do it? where do i put it? and whats best to do? send them to the custom 404 page or the index page :(
    sammie xox
     
    samantha pia, Jan 11, 2005 IP
  3. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #43
    As I understand it, you've now renamed the FAQ page to something else, correct?

    First, re-upload that robots.txt page to exclude spiders from indexing the pages you don't want indexed. Then we can check it for errors.

    Second, to direct to a custom 404 error page, you need an entry in your .htaccess file -- if your host allows this. It would look like this:

    ErrorDocument 404 /error.htm
    Code (markup):
    where for "error.htm" you would substitute whatever you've called your custom error page.

    If your host doesn't allow you to edit the .htaccess file (if you're using FrontPage with FP extensions, be careful: this line needs to be added to the end of the existing file), or if you're not on a *nix/Apache server, then you should ask the host to enable the custom error page for you.

    But note that of the "error hits" you mention:

    /v-web/bulletin/bb/faq.php 615 - 
    /v-web/bulletin/bb/viewtopic.php 127 
    /v-web/bulletin/bb/viewtopic.php?p=132&sid=76f314b903e458a2da56cb7b3926aea1 
    Code (markup):
    the 2nd and 3rd ones are looking for specific forum posts, possibly posts that have been deleted?
     
    minstrel, Jan 11, 2005 IP
  4. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #44
    i have access to the .htaccess files. google bot never came back once i put the robot.txt file on the server, i have installed a newer phpbb forum but not in the old v-web dir where the old one was, and has now been moved and stored on the server, but not in use anymore. the .htaccess file is full of
    and this one
    can i just redirect all of them to the new forums?
     
    samantha pia, Jan 11, 2005 IP
  5. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #45
    Sure -- redirect them to the new forum and then put in a new 404 page and a new robots.txt file.
     
    minstrel, Jan 11, 2005 IP
  6. phrozen_ra

    phrozen_ra Peon

    Messages:
    147
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #46
    Had a forum once.... it was stuck on my pages too... at least that's why i believed... but NO

    But tell me this... are you running google ads on your site? if so... then it's not the google bot, but the bot that figures out what kind of content you have

    If I am right... then you can filter it out of your logs... don't... by any means, don't boot it
     
    phrozen_ra, Jan 12, 2005 IP
  7. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #47
    how? i tried to do the redirect by adding this line: Redirect 301 /v-web/bulletin/bb/faq.php http://www.asksam2.com/index.php just to send them back to the main page of the site, but there are 2 buttons in the .htaccess thingy, one says update file, the other says reset. i hit the update one, and nothing happens, :( and i think if i hit the reset one, it will deleted all the stuff in it :( so i aint tried that yet. :eek:


    and i dont have any ads on my site what so ever, so it was just the googlebot stuck for 3 days, i just keep getting 404's now from the old forums, 2,400+ to date, thats 800 in one day.
     
    samantha pia, Jan 12, 2005 IP
  8. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #48
    Sounds like you're using CPanel to do this. Contact your host to ask why it isn't working.
     
    minstrel, Jan 12, 2005 IP
  9. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #49
    the problem is it is working with the other 100/150 301 redirects, its just not doing it with the new one i tried to do, it shows an old custom 404 page. i guess its in a script somewhere, thats doing it
     
    samantha pia, Jan 12, 2005 IP
  10. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #50
    Yikes. Maybe there's a limit? All those redirects -- I'm no expert on servers but that must be slowing things down a tad... why do you need so many?
     
    minstrel, Jan 12, 2005 IP
  11. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #51
    the old site and new site work diffrently, the old one was just html, with a java left hand menu, the new site is all php, and the new site has been setup much better than the old site, so to carry on getting the old sites SE traffic, each old page was redirected to the new page, after a few months of this, i think it would be safe to delete the old site, but the new site has new forums and chatroom and could not use the old v-web and had to be placed in a dir in another place on the site.
     
    samantha pia, Jan 12, 2005 IP
  12. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #52
    You can replace the 150 lines with just one.

    Put this line in tyhe .htaccess file at the old site:

    redirect 301 / http://www.newsite.com/
    Code (markup):
    where "http://www.newsite.com/" is of course replaced with the URL for your new site.
     
    minstrel, Jan 12, 2005 IP
  13. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #53
    the new site was rebuilt 6 weeks ago, the old site is still on the same server, the url is the samefor both, but the index page is now for the new site, but the old sites pages still get hits, so they are redirected to the new pages in the new dir's for the new site.
     
    samantha pia, Jan 12, 2005 IP
  14. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #54
    i know i am hard work. wanna see inside my head? not much in there you know, lol
    sorry if i am not explaining it right, its just so confusing.
     
    samantha pia, Jan 12, 2005 IP
  15. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #55
    :D My point is the phenomenon you saw where Googlebot seemed to be trapped may have something to do all those redirects and having it run back and forth and everywhere.

    1. delete all the redirects and just put in the one above -- that will redirect all requests for an old forum page to the new forum -- then you can delete the old forum
    2. get someone to help you update your forum to 2.0.11 -- I know the upgrade installation program is buggy but if you don't it's only a matter of time until you lose your forum
    3. once it's installed, delete the install folder and the contribute folder
     
    minstrel, Jan 12, 2005 IP
  16. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #56
    ok minstel, i cant redirect them like you said, but taking in what you said, made me thing big time, enough to get a headache. but i found a way round the redirect problem with them, ie i get 4 main 404's from the following links:
    /v-web/bulletin/bb/faq.php 627 404's
    /v-web/bulletin/bb/login.php 388 404's
    /v-web/bulletin/bb/viewtopic.php 127 404's
    /v-web/bulletin/bb/index.php 392 404's
    /v-web/bulletin/bb/profile.php 65 404's

    now i fixed them all, i fixed them by placing a new site page on female sexuality with the main and sub menus on it, so people will see the following if they enter the site via any of the above 404's
    http://asksam2.com/v-web/bulletin/bb/viewtopic.php

    but without your help, i would never have sat down all day and thought about it, and i am very chuffed i fixed it,
    so a great be kiss and thank you form sammie xoxox
     
    samantha pia, Jan 13, 2005 IP
  17. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #57
    Uh-oh. I'm really sorry about the headache -- not what I intended but apparently I've had that effect on women before :D

    I'm not sure I understand why you can't do the one redirect but if it's working, that's what matters, right?

    Good luck with Googlebot in the future :eek:

    And thanks for the Sammie kiss :D
     
    minstrel, Jan 13, 2005 IP
  18. minstrel

    minstrel Illustrious Member

    Messages:
    15,082
    Likes Received:
    1,243
    Best Answers:
    0
    Trophy Points:
    480
    #58
    Actually, looking at that "Sammie kiss" thing, I'm reminded of a website a friend created called "Sammy Pies" -- they are dog treats :eek:
     
    minstrel, Jan 13, 2005 IP
  19. samantha pia

    samantha pia Prominent Member

    Messages:
    4,639
    Likes Received:
    482
    Best Answers:
    0
    Trophy Points:
    310
    #59
    lol i just love giving kisses [​IMG]
     
    samantha pia, Jan 22, 2005 IP