cURL Post Question

So, im trying to learn cURL to post forms, etc.

Ive read a couple tutorials and it seems like some add the form submit to the post string and some dont.

I imagine you would need to, otherwise it wont actually complete posting the form. However, what is the correct format for doing so?

Say you have <input type=submit name=btnSubmit value=Submit>

Would the cURL post for that look like "....restofposthere....&btnSubmit=Submit"

What if the btnSubmit doesnt have a value specified, how do you post that?

 

The value of btnSubmit is "Submit". However, if a field doesn't have a value, you can just leave it blank:

foo=&bar=&foo1=bar1&...

 

Im sure this is a broad question, but does anyone know if its possible to login to CJ.Com via cURL?

I found the login URL which is https://members.cj.com/member/foundation/memberlogin.do

And it looks like it logs me in correctly but it says I need to transferred somewhere else and gives me a link on the page but doesnt redirect via cURL

 

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

PHP:

 

It wont let me use that command, its complaining about safe mode or opendir base...

I managed to get in by using the home address and using a session id it generated... it looks like: https://members.cj.com/member/publisher/home.do;jsessionid=sessidhere

That gets me to my homepage, ultimately I would like to get to a product page, the main goal is to be able to pull products into a format that I can insert into a database..

Please be aware, this isnt mindless scraping, I have specific searches that still encompass a wide array of products...

 

Ah, you should enable cookies as well.

curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookies.txt');

PHP:

 

I did already, I guess my question is, is it possible to navigate web pages with cURL, or is it more useful for posting forms, etc.