we are doing a new site now that has a members only section .we want to know how to get the search engines to those pages even though you need a user name and password to get to them.
build duplicate pages if it's database driven and link on your sitemap. Just make sure you don't require the sessions or cookies on the new pages. If you are not willing to do that, filtering by ip address is the only way to go but this is a close relative of cloaking. Tread with caution
I think you still have to separate between free content and members only content. If you allow google bot to index your members only content, then your membership would be useless since anyone can see the content of your pages on google cache.
I've tried visiting news articles links I found listed in Google, only to find a regestration was required to read article. Free, but still a blockade. I've always kinda wondered how they got listed. I figured it must have been some kind of javascript thing, that noticed you weren't logged in, but google ignored.
I believe it was webmasterworld . com that was famous for it, I think they have since changed their policies. The trick is to display a different page to spiders than you do to normal users. This is commonly used to show spiders information for SERPs and then redirect visitors to advertising pages, it's called cloaking. Personally I would rather see a totally spammy page show up when I click a link in an engine than have it send me to a "pay to register" page. At least with spam I can see it and hit the back button in a hurry. If I really want the information on the page I have to think for a sec how much that info is worth to me before hitting back and finding it somewhere else. :] By the way, you can specifically tell bots not to cache your page. What this will do is just display a little teaser in the SERPs and have 99% of the page require registration. If the link is cached you can click the link in google and see the whole page as googlebot cached it. If engines don't cache the page, only the snippet will show up giving searchers a teaser as to what to expect on the page.