if in a site, there are some sections which can be accessed only after login. Does crawler crawl this sections? is there anyway to make this happen? I mean have crawlers crawl the sections behind login. if yes how will that be done becuase when they try to index the link of article detail page, it will be directed to login page. so how will it be done Regards, Nitin
Nope - search engines will not crawl content behind login forms (because they would need to login first!). If you want content to be publicly accessible then I'd recommend you put that content before the login form. If you try doing clever stuff to let Google spider content behind your login form then you're going to create loads of problems for yourself!
carleisenstein is correct. Crawlers cannot get to pages only viewable behind a login to index them. They cannot log into each site to index this kind of content because that would require the crawler to have a username/password. So you don't have to worry about optimizing content behind your login screen for search engines.
I would really not suggest to do something which will create banning for you. Why don’t you keep before login. Google spider is very intelligent.