Hello! I was wondering if someone knew 2 things about cURL: 1) if posting to a form with no action using cURL matters. <form method=post> Code (markup): There are no tokens created or hidden fields with session variables... this should be a straight forward submit. 2) and how to handle field names with forward slashes in them I am building an array and have wrapped the field names with slashes in quotes" $ps["State/Province"] = $state; $ps["Zip/Postal Code_prompt"] = 'Zip/Postal Code'; Code (markup): but fields without "/" I add to the array like this: $ps[Country] = $country; Code (markup): Does this look like a problem to anyone? I am basically trying to submit all this data to a page and it is just coming back with the empty form with no errors... I do an echo like this: $x = curl_exec($ch); echo $x; Code (markup): and it comes back with the blank form and of course no data because it never makes it to that second and final page. I have successfully completed the sign up process manually and have extracted the data I need out of the static page (using explode) so that is not a problem, it just seems like the post is not initiating. I seriously appreciate any help you could offer. Thanks, Mitch
i think when building array u must define by ID .. this is just example.. no matter if it doesnt hv action. ..it will execute php self state is define by ID . usually place in input TAG
Oh sorry ... r u trying to Submit with cURL to php self? i fu do ... it still can buat u must define URL target ...
Thanks for the response... Here's my code... It just returns the blank form and never submits to the form.. $ps[FirstName_prompt] = 'First Name'; $ps[FirstName] = $name; $ps[LastName_prompt] = 'Last Name'; $ps[LastName] = $name; $ps[Email_prompt] = 'Email'; $ps["State/Province"] = $state; $ps["Zip/Postal Code_prompt"] = 'Zip/Postal Code'; $ps["Zip/Postal Code"] = $zip; etc......... $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://www.thedomain.com/signup/?aid='.$ps[parent_id]); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $ps); curl_setopt($ch, CURLOPT_FOLLOWLOCATION ,1); curl_setopt($ch, CURLOPT_COOKIEFILE, 'tmp/cookie'); curl_setopt($ch, CURLOPT_COOKIEJAR, 'tmp/cookie'); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); //echo "test"; //echo $ps[FirstName]; $x = curl_exec($ch); echo $x; Code (markup): Am i missing something? Remember the form submits to itself... Thanks again for taking a look at this! Mitch
Try: curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($ps, '', '&')); PHP: ... submitting the data in type of array, with specific keys will force the script to send the data encoded as multipart/form-data. And I don't see $ps[parent_id] being set anywhere. And you might as well try setting a user agent.
Thanks a lot for that, I am seeing some progress now... I did everything you said and now the form comes back with the fields filled this time instead of blank... though It's still not submitting for some reason... I noticed that when the data is displayed in the fields a couple of required elements were not selected (the country and the "agree to" checkbox)... so I am messing with that now. And yes I set $ps[parent_id] above in the code... So it might be this checkbox and country fields... Here is the header info... I was wondering if you could spot anything useful in here... HTTP/1.1 100 Continue HTTP/1.1 100 Continue HTTP/1.1 200 OK Date: Fri, 22 Aug 2008 23:30:34 GMT Server: Apache/2.2.9 (Unix) mod_ssl/2.2.9 OpenSSL/0.9.8g mod_bwshare/0.2.1 P3P: policyref="https://www.domain.com/w3c/p3p.xml", CP="NOI DSP COR CURa OUR BUS NAV STA" Content-Type: text/html Set-Cookie: AffiliateLink=aid|54392||mid|; expires=Sat Aug 23 23:30:34 2008; path=/al/; Transfer-Encoding: chunked Code (markup): Thanks again for your responses! Mitch
So the only required element this is not active is the agree to checkbox. This is the code for the checkbox in the form I am trying to submit: <input type="checkbox" name="accept" value="1"> Code (markup): So how can I check this when submitting with cURL? Thanks!!
$ps['accept'] = 1; PHP: EDIT: This might as well needs to be a valid email address (or at least a valid format): $ps[Email_prompt] = 'Email'; PHP:
I have always had trouble with cURL, so I wrote my own wrapper that uses fsockopen. If you want to find the postdata, download the tremendous firefox addon, Live HTTP Headers. Additionally, if you there is no form action, then the form action is simply the same page the form is being posted on. For example, if you have a file "page.php", and form like this refresh the page. <form action="" method="post"><input type="submit" value="Refresh!" /></form> Code (markup): If you want to check out my wrapper, here it is: http://rcadble.net/viewcode.php?id=20 Internet wrapper class using sockets Added Sunday, 3rd August 2008 Information Language: php Code Type: class Description This is a PHP5 class which can be used to access the internet and make requests. There are comments all the methods. Features: * GZip Compressions * Optional custom headers * Optional custom user agent * Auto parse cookies to a 2d array (hosts, names) * Support for use of proxies * Option for automatically following redirects * Get or set cookies by host name and cookie name * Function to clear all cookies from all hosts, or all cookies by host name * GetBetween function you can use to parse data Source Code (316 lines): /* * PHP fsockopen internet wrapper by rcadble (services@rcadble.net) * Features G-Zip, custom headers or user agent, a 2d array for cookies, follow redirect, etc * There are brief descriptions for each method * Feel free to redistribute this wrapper as long as these comments remain intact */ class sWrapper { private $cookies = array(); private $lastPage = NULL; private $userAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0"; private $proxyHost = NULL; private $proxyPort = NULL; private $useProxy = false; private $useGZip = true; private $redirect = false; /* * Enables automatic redirect follow (redirect is disabled by default) */ public function enableRedirect() { $this->redirect = true; } /* * Disables automatic redirect follow (redirect is disabled by default) */ public function disableRedirect() { $this->redirect = false; } /* * Stops GZip in requests (GZip is enabled by default) */ public function disableGZip() { $this->useGZip = false; } /* * Enables GZip in requests (GZip is enabled by default) */ public function enabledGZip() { $this->useGZip = true; } /* * Sets the proxy host and port. * Note: this does not turn on or off usage of a proxy. */ public function setProxy($host, $port) { $this->proxyHost = $host; $this->proxyPort = $port; } /* * Stops usage of proxy */ public function stopProxy() { $this->useProxy = false; } /* * Starts usage of proxy */ public function startProxy() { $this->useProxy = true; } /* * Returns the current set proxy in the format of host:port */ public function getProxy() { return $this->proxyHost . ":" . $this->proxyPort; } /* * Sets the user agent to the value of your choice. */ public function setUserAgent($newAgent) { $this->userAgent = str_replace(array("User-Agent: ", "\r\n"), "", $newAgent); } /* * Returns an array with all the cookies from a given domain name * i.e. $cookies = $wrapper->getCookiesByHost("www.rcadble.net") */ public function getCookiesByHost($host) { $hostCookies = array(); if (count($this->cookies[$host])!=0) { foreach ($this->cookies[$host] as $cookiename => $cookievalue) $hostCookies[$cookiename] = $cookievalue; } return $hostCookies; } /* * Returns a 2d array with all the cookies stored in the class * The first dimension is the host name, the second is the cookie name */ public function getAllCookies() { return $this->cookies; } /* * Gets a particular cookie with a given name from a given domain name * i.e. $cookievalue = $wrapper->getCookiesByName(www.youtube.com, "LOGIN_INFO"); */ public function getCookieByName($host, $name) { return $this->cookies[$host][$name]; } /* * Sets a cookie with a given host and cookie name */ public function setCookie($host, $name, $value) { $this->cookies[$host][$name] = $value; } /* * Clears all cookies */ public function clearCookies() { $this->cookies = array(); } /* * Clears all cookies from specified host */ public function clearCookiesByHost($host) { unset($this->cookies[$host]); //$this->cookies = array_values($this->cookies); } /* * Clears a specific cookie from a given host and name */ public function clearCookieByName($host, $name) { unset($this->cookies[$host][$name]); } /* * Gets the current set userAgent. */ public function getUserAgent() { return $this->userAgent; } /* * Returns the location of the last page requested through the wrapper. */ public function getLastPage() { return $this->lastPage; } /* * Makes an HTTP request * $method should be set to either "GET" or "POST" * $destnation is the page to request * optional $referrer is the page to appear that you've requested from * optional $postdata is data posted during the request * optional $showHeaders is whether or not to return the resulting headers * optional $custHeaders is an assosiative array containing the headers you want to request with */ public function request($method, $destination, $referrer = "lastpage", $postdata = "", $showHeaders = true, $custHeaders = array()) { if ($referrer=="lastpage") $referrer = "Referrer: " . $this->lastPage . "\r\n"; elseif ($referrer!="") $referrer = "Referrer: {$referrer}\r\n"; $method = strtoupper($method); $host = preg_replace("/\/.*/", "", str_replace("http://", "", $destination)); if (strpos($host, "www.")===false) $host = "www." . $host; $path = str_replace("http://{$host}", "", $destination); if ($path{0}!="/") $path = "/{$path}"; if ($this->useProxy===true) $fp = fsockopen($this->proxyHost, $this->proxyPort, $errornum, $errorstr, 20); else $fp = fsockopen($host, 80, $errornum, $errorstr, 20); if ($fp) { if (count($this->cookies[$host])!=0) $assembledCookies = "Cookie: " . $this->assembleCookies($host) . "\r\n"; else $assembledCookies = ""; if (count($custHeaders)!=0) { if ($this->useProxy!==true) { $header = "{$method} {$path} HTTP/1.1\r\n"; foreach ($custHeaders as $headername => $headervalue) { if ($headername=="Cookie") { $header .= $assembledCookies; } else { if (strpos($headervalue, "\r\n")!==false) $header .= "{$headername}: {$headervalue}"; else $header .= "{$headername}: {$headervalue}\r\n"; } } } else { $header = "{$method} http://{$host}{$path} HTTP/1.1\r\n"; foreach ($custHeaders as $headername => $headervalue) { if ($headername=="Host") { $header .= "Host: {$this->proxyHost}\r\n"; } elseif ($headername=="Cookie") { $header .= $assembledCookies; } else { if (strpos($headervalue, "\r\n")!==false) $header .= "{$headername}: {$headervalue}"; else $header .= "{$headername}: {$headervalue}\r\n"; } } } if ($postdata!="") { parse_str($postdata, $formdata); $postdata = http_build_query($formdata); if (strpos($header, "Content-Type:")===false) $header .= "Content-Type: application/x-www-form-urlencoded\r\n"; if (strpos($header, "Content-Length:")===false) $header .= "Content-Length: " . strlen($postdata) . "\r\n"; if (strpos($header, "Connection:")===false) $header .= "Connection: Close\r\n\r\n"; $header .= $postdata; } else { if (strpos($header, "Connection: Close\r\n\r\n")===false) $header .= "Connection: Close\r\n\r\n"; } } else { if ($this->useProxy===true) { $header = "{$method} http://{$host}{$path} HTTP/1.1\r\n"; $header .= "Host: {$this->proxyHost}\r\n"; } else { $header = "{$method} {$path} HTTP/1.1\r\n"; $header .= "Host: {$host}\r\n"; } $header .= "User-Agent: {$this->userAgent}\r\n"; $header .= "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n"; $header .= "Accept-Language: en-us,en;q=0.5\r\n"; if ($this->useGZip===true) $header .= "Accept-Encoding: gzip,deflate\r\n"; $header .= "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n"; $header .= "Keep-Alive: 300\r\n"; if ($referrer!="") $header .= $referrer; $header .= $assembledCookies; if ($postdata!="") { parse_str($postdata, $formdata); $postdata = http_build_query($formdata); $header .= "Content-Type: application/x-www-form-urlencoded\r\n"; $header .= "Content-Length: " . strlen($postdata) . "\r\n"; $header .= "Connection: Close\r\n\r\n"; $header .= $postdata; } else { $header .= "Connection: Close\r\n\r\n"; } } fwrite($fp, $header); $data = null; while (!feof($fp)) $data .= fgets($fp); fclose($fp); $this->lastPage = $destination; $this->parseCookies($this->getBetween($data, "HTTP", "Connection: close"), $host); if (strpos($data, "Transfer-Encoding: ")!==false) $info['transfer-encoding'] = $this->getBetween($data, "Transfer-Encoding: ", "\r\n"); if (strpos($data, "Content-Encoding: ")!==false) $info['content-encoding'] = $this->getBetween($data, "Content-Encoding: ", "\r\n"); $content = split("\r\n\r\n", $data, 2); if ($showHeaders) $data = $content[0] . "\r\n" . $this->decodeBody($info, $content[1]); else $data = $this->decodeBody($info, $content[1]); if ($this->redirect===true&&strpos($content[0], "Location: ")!==false) { if ($c = preg_match_all('/Location:\\s+((?:http|https)(?::\\/{2}[\\w]+)(?:[\\/|\\.]?)(?:[^\\s"]*))/is', $content[0], $matches)) { $data = $this->request("GET", $matches[1][0], $destination, "", $showHeaders); } } return $data; } else { return false; } } /* * Returns the text between two strings $startstring and $endstring located in string $source starting at character $start */ public function getBetween($source, $startstring, $endstring, $start = 0) { return substr($source, strpos($source, $startstring, $start) + strlen($startstring), strpos($source, $endstring, $start) - strlen($startstring) - strpos($source, $startstring, $start)); } /* * Private method which assembles cookies to a transmittable string */ private function assembleCookies($host) { foreach ($this->cookies[$host] as $cookieName => $cookieValue) $assembled .= "{$cookieName}={$cookieValue}; "; return $assembled; } /* * Private method which parses cookies to a 2d assosiative array */ private function parseCookies($header, $defaultHost) { $lines = split("\r\n", $header); foreach ($lines as $line) { if (strpos($line, "domain=.")!==false) $host = "www" . substr($line, strpos($line, "domain=") + strlen("domain="), strlen($line) - (strpos($line, "domain=") + strlen("domain="))); else $host = $defaultHost; if (preg_match("/Set-Cookie: (.+?)=(.+?);/", $line, $matches)) $this->cookies[$host][$matches[1]] = $matches[2]; } } /* * Private method which decodes the gzip-deflated content * I modified this function to suit my needs but I did not originally create it */ private function decodeBody($info, $content) { $add = strlen("\r\n"); if (isset($info['transfer-encoding']) && $info['transfer-encoding'] == 'chunked') { do { $content = ltrim($content); $pos = strpos($content, "\r\n"); $len = hexdec(substr($content, 0, $pos)); if (isset($info['content-encoding'])) $decoded .= gzinflate(substr($content, ($pos + $add + 10), $len)); else $decoded .= substr($content, ($pos + $add), $len); $content = substr($content, ($len + $pos + $add)); $check = trim($content); } while (!empty($check)); } else if (isset($info['content-encoding'])) { $decoded = gzinflate(substr($content, 10)); } else { $decoded = $content; } return $decoded; } } Code (markup):