curl Time Out Problem and Solution

If you want cURL to timeout in less than one second, you can use CURLOPT_TIMEOUT_MS, although there is a bug/”feature”  on “Unix-like systems” that causes libcurl to timeout immediately if the value is < 1000 ms with the error “cURL Error (28): Timeout was reached”.  The explanation for this behavior is:

“If libcurl is built to use the standard system name resolver, that portion of the transfer will still use full-second resolution for timeouts with a minimum timeout allowed of one second.”

What this means to PHP developers is “You can use this function without testing it first, because you can’t tell if libcurl is using the standard system name resolver (but you can be pretty sure it is)”

The problem is that on (Li|U)nix, when libcurl uses the standard name resolver, a SIGALRM is raised during name resolution which libcurl thinks is the timeout alarm.

The solution is to disable signals using CURLOPT_NOSIGNAL.  Here’s an example script that requests itself causing a 10-second delay so you can test timeouts:

<?php
if (!isset($_GET['foo'])) {
// Client
$ch = curl_init('http://localhost/test/test_timeout.php?foo=bar');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_NOSIGNAL, 1);
curl_setopt($ch, CURLOPT_TIMEOUT_MS, 200);
$data = curl_exec($ch);
$curl_errno = curl_errno($ch);
$curl_error = curl_error($ch);
curl_close($ch);

if ($curl_errno > 0) {
echo "cURL Error ($curl_errno): $curl_error\n";
} else {
echo "Data received: $data\n";
}
} else {
// Server
sleep(10);
echo "Done.";
}
?>

Lets Begin curl Scripting

cURL is a library which allows you to connect and communicate to many different types of servers with many different types of protocols. Using cURL you can:

  • Implement payment gateways’ payment notification scripts.
  • Download and upload files from remote servers.
  • Login to other websites and access members only sections.

PHP cURL library is definitely the odd man out. Unlike other PHP libraries where a whole plethora of functions is made available, PHP cURL wraps up major parts of its functionality in just four functions.

A typical PHP cURL usage follows the following sequence of steps.

curl_init – Initializes the session and returns a cURL handle which can be passed to other cURL functions.

curl_opt – This is the main work horse of cURL library. This function is called multiple times and specifies what we want the cURL library to do.

curl_exec – Executes a cURL session.

curl_close – Closes the current cURL session.

Below are some examples which should make the working of cURL more clearer.

Download file or web page using PHP cURL

The below piece of PHP code uses cURL to download Google’s RSS feed.

<?php
//Initialize the cURL session
$ch = curl_init();
//Set the URL of the page or file to download.
curl_setopt($ch, CURLOPT_URL, 'http://news.google.com/news?hl=en&topic=t& output=rss');
//Ask cURL to return the contents in a variable instead of simply echoing them to the browser.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Execute the cURL session
$contents = curl_exec ($ch);
//Close cURL session
curl_close ($ch);
?>

As you can see, curl_setopt is the pivot around which the main cURL functionality revolves. cURL functioning is controlled by way of passing predefined options and values to this function.

The above code uses two such options.

  • CURLOPT_URL: Use it to specify the URL which you want to process. This could be the URL of the file you want to download or it could be the URL of the script to which you want to post some data.
  • CURLOPT_RETURNTRANSFER: Setting this option to 1 will cause the curl_exec function to return the contents instead of echoing them to the browser.

Download file or web page using PHP cURL and save it to file

The below PHP code is a slight variation of the above code. It not only downloads the contents of the specified URL but also saves it to a file.

<?php
//Initialize the cURL session
$ch = curl_init();
//Set the URL of the page or file to download.
curl_setopt($ch, CURLOPT_URL, 'http://news.google.com/news?hl=en&topic=t& output=rss');
//Create a new file
$fp = fopen('rss.xml', 'w');
//Ask cURL to write the contents to a file
curl_setopt($ch, CURLOPT_FILE, $fp);
//Execute the cURL session
curl_exec ($ch);
//Close cURL session and file
curl_close ($ch);
fclose($fp);
?>

Here we have used another of the cURL options, CURLOPT_FILE. Obtain a file handler by creating a new file or opening an existing one and then pass this file handler to the curl_set_opt function.

cURL will now write the contents to a file as it downloads a web page or file.