Crawling restricted area

Hi,
I have certain ares in my site which can be viewed only by registered members. Will google crawl those areas. If not how to instruct the robots to do so.

 

if those areas are accessible only by registered members then it means that to access that area a user id and password is needed which means robots can't crawl those pages hence cannot be crawled.

 

Is there a way to allow the robots to crawl, while neglecting access to guests.

 

You can do it using website cloaking.. however, I would suggest you not to use this technique as it could lead your website to a ban over Google..

 

Then it is a absolute no no. Are there any legal ways.

 

There is no legal way, however, you can send a formal e-mail seeking permission of applying Cloaking on your website from Google...

But I am not pretty sure, whether they will reply or not??

 

Ok then when my site has one million views i will apply and google will reply.

 

All depends on Google mood

 

If you want those pages to be crawled by Google, you can set the authorization level of "View" for all. It means even the guest user can view the page, but for any changes he will have to provide the User ID and Password. and when the user can view the page then crawlers can read that page.

 

Question is somewhat different...as he wants to know, how can crawlers view the sections that could be viewed only by registered users.
as far i Know, crawlers couldn't provide user name and password so they couldn't view those sections.. to do that, one has to apply cloaking..

 

Many thanks for your reply but manish.chauhan is right.

 

Hi,
Now with another question.
Will the following be possible.

1. I will set access to certain pages such that they are accessible only to registered users as said above.


Will it be possible for the search engines to see the meta tags at least.

I need members and i need users from search engines too!!

 

No...There is no technique(except cloaking) in my knowledge that can give access to crawlers to see any password protected web page...

 

Thank you Manish. Let me now if you get to know some thing.

Would love to give rep but not possible for me right now (Damn newbie me).


Shailendra saw your name in searchcamp.in.

 

Whats searchcamp.in??

 

A conference on search marketing. People from the industry meet and network. I have not attended any search camps and for more details http://searchcamp.in/


Hope you can attend. Do share how it went

 

Thanks for this info..I'll try to attend it..