can spider enter sites which require password?

Discussion in 'Google' started by dadaas, Jul 27, 2008.

  1. #1
    Ok, i need help, do i need to do soemthign special to allow google crawl trough my site, site is www.spacehumor.com and its very new site, problem is to see joke you need to be lofged in, so how can i log in google so they can crawl site?

    I think i have read about this somewhere but i forgot where.
     
    dadaas, Jul 27, 2008 IP
  2. JoX23

    JoX23 Peon

    Messages:
    60
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Google has a program called first click free where you configure your webserver to not show google the registration page and they will index your password protected content. I don't remember much about it but I am sure if you do a search in google help you can find it.
     
    JoX23, Jul 27, 2008 IP
  3. eXe

    eXe Notable Member

    Messages:
    4,643
    Likes Received:
    248
    Best Answers:
    0
    Trophy Points:
    285
    #3
    You need to fix it so that any visitor with useragent as googlebot can view the page. This may attract a penalty if you don't implement google's "first click free" policy, which defeats the purpose. Combine with nocache to ensure it isn't possible for a surfer to view your joke by just accessing google's cached version.
     
    eXe, Jul 27, 2008 IP
  4. dadaas

    dadaas Well-Known Member

    Messages:
    1,298
    Likes Received:
    17
    Best Answers:
    0
    Trophy Points:
    160
    #4
    i hope they accept me

    well i ahve send them request for that but my website is still not idnexing jokes www.spacehumor.com

    please help me im very confused about hwo to make google index my site.
     
    dadaas, Jul 30, 2008 IP
  5. pubdomainshost.com

    pubdomainshost.com Peon

    Messages:
    1,277
    Likes Received:
    21
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Google Analytics does have a section where you can provide user credentials to allow google to crawl your site (after login) ... I suggest you check Google Webmaster help to learn more about the feature.

    Also it takes a while for results to show up .. so you may have to be bit patient as well

    Thx
     
    pubdomainshost.com, Aug 29, 2008 IP
  6. expiringdomains

    expiringdomains Peon

    Messages:
    786
    Likes Received:
    11
    Best Answers:
    0
    Trophy Points:
    0
    #6
    It will surely be indexed in a short span of time. Just be patient.

    If you have allowed and authorized the bot to enter appsword protected pages yes they can crawl and index those pages if not then it might be difficult for a bot to reach there.
     
    expiringdomains, Aug 29, 2008 IP
  7. alemcherry

    alemcherry Guest

    Best Answers:
    0
    #7
    First click free is, I guess applicable only for google news. I am not sure about it, but probably showing content to googlebot and not to the visitors might fall under clocking. You are taking a bit of risk in doing so.

    In my opinion, it is a lot better if you open up your site and allow visitors to see the content without requiring registration. Jokes is an over populated niche and there are tons of free to all high quality sites available. As a newbie, it doesnt make sense for you to hide your content. Registration and login is for sites that has something confidential - or very high quality content. Atleast 90% of your visitors would leave the site the moment they see a registration page.

    My 2 cents, up to you to decide.
     
    alemcherry, Aug 29, 2008 IP
  8. dadaas

    dadaas Well-Known Member

    Messages:
    1,298
    Likes Received:
    17
    Best Answers:
    0
    Trophy Points:
    160
    #8
    yea but i dont know how to remove login form from that script. maybe i need to go to other section of forum for this help.
     
    dadaas, Aug 29, 2008 IP
  9. koothichi

    koothichi Banned

    Messages:
    5
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Ya SPIDER CAN ENTER ANY WHERE WITH OUT ANY PASSWORD..........
     
    koothichi, Aug 29, 2008 IP
  10. rehash

    rehash Well-Known Member

    Messages:
    1,502
    Likes Received:
    30
    Best Answers:
    0
    Trophy Points:
    150
    #10
    you know that ANYONE can run a spider, don't you? so think again
     
    rehash, Aug 29, 2008 IP