wordpress / requests

Requests for PHP is a humble HTTP request library. It simplifies how you interact with other sites and takes away all your worries.

Requests for PHP

Requests is a HTTP library written in PHP, for human beings. It is roughly based on the API from the excellent Requests Python library. Requests is ISC Licensed (similar to the new BSD license) and has no dependencies, except for PHP 5.6.20+.

Despite PHP's use as a language for the web, its tools for sending HTTP requests are severely lacking. cURL has an interesting API, to say the least, and you can't always rely on it being available. Sockets provide only low level access, and require you to build most of the HTTP response parsing yourself.

We all have better things to do. That's why Requests was born.

$headers = array('Accept' => 'application/json');
$options = array('auth' => array('user', 'pass'));
$request = WpOrg\Requests\Requests::get('https://api.github.com/gists', $headers, $options);

var_dump($request->status_code);
// int(200)

var_dump($request->headers['content-type']);
// string(31) "application/json; charset=utf-8"

var_dump($request->body);
// string(26891) "[...]"

Requests allows you to send HEAD, GET, POST, PUT, DELETE, and PATCH HTTP requests. You can add headers, form data, multipart files, and parameters with basic arrays, and access the response data in the same way. Requests uses cURL and fsockopen, depending on what your system has available, but abstracts all the nasty stuff out of your way, providing a consistent API.

Features

• International Domains and URLs
• Browser-style SSL Verification
• Basic/Digest Authentication
• Automatic Decompression
• Connection Timeouts

Installation

Install with Composer

If you're using Composer to manage dependencies, you can add Requests with it.

composer require rmccue/requests

or

{
    "require": {
        "rmccue/requests": "^2.0"
    }
}

Install source from GitHub

To install the source code:

$ git clone git://github.com/WordPress/Requests.git

Next, include the autoloader in your scripts:

require_once '/path/to/Requests/src/Autoload.php';

You'll probably also want to register the autoloader:

WpOrg\Requests\Autoload::register();

Install source from zip/tarball

Alternatively, you can fetch a tarball or zipball:

$ curl -L https://github.com/WordPress/Requests/tarball/stable | tar xzv
(or)
$ wget https://github.com/WordPress/Requests/tarball/stable -O - | tar xzv

Using a Class Loader

If you're using a class loader (e.g., Symfony Class Loader) for PSR-4-style class loading:

$loader = new Psr4ClassLoader();
$loader->addPrefix('WpOrg\\Requests\\', 'path/to/vendor/Requests/src');
$loader->register();

Documentation

The best place to start is our prose-based documentation, which will guide you through using Requests.

After that, take a look at the documentation for \WpOrg\Requests\Requests::request(), where all the parameters are fully documented.

Requests is 100% documented with PHPDoc. If you find any problems with it, create a new issue!

Test Coverage

Requests strives to have 100% code-coverage of the library with an extensive set of tests. We're not quite there yet, but we're getting close.

Requests and PSR-7/PSR-18

PSR-7 describes common interfaces for representing HTTP messages. PSR-18 describes a common interface for sending HTTP requests and receiving HTTP responses.

Both PSR-7 as well as PSR-18 were created after Requests' conception. At this time, there is no intention to add a native PSR-7/PSR-18 implementation to the Requests library.

However, the amazing Artur Weigandt has created a package, which allows you to use Requests as a PSR-7 compatible PSR-18 HTTP Client. If you are interested in a PSR-7/PSR-18 compatible version of Requests, we highly recommend you check out this package.

Contribute

Contributions to this library are very welcome. Please read the Contributing guidelines to get started.

requests's Issues

Support parallel requests with sockets

@rdlowrey's Artax client supports parallel requests with sockets. Currently, this is the only real difference between cURL and sockets in terms of functionality.

We should investigate whether proper parallelisation can be supported with sockets, and if so, what the requirements are for this. Alert is listed as requiring 5.4+, but I'm not sure on whether this is just Alert or if event reactors aren't possible on 5.2/5.3. In either case, I'm in favour of supporting parallelisation on the platforms that support it and falling back if not, as it's still an upgrade from what we have now.

GET request query strings in $data paramater

First off, great library. It's really simple and helping me a lot so far.
I first set up a POST request and it worked as expected but got tripped up on GET requests.

On line 281 of Requests.php, $data is defined as:

@param array $data Data to send either as a query string for GET/HEAD requests, or in the body for POST requests

Based on this, one would expect the get function on line 193 of Requests.php to take $data as an input parameter but it doesn't.

public static function get($url, $headers = array(), $options = array()) {
    return self::request($url, $headers, null, self::GET, $options);
}

In fact it sets what the request function on line 286 expects as data to null.

If I use the following code:

$request = Requests::get($url . '?' . $query, $headers, $options);

or if I just include my $query in my $url variable then this works as expected. Is this how GET requests are supposed to be handled?

Wouldn't it make more sense to use $data the same way as you have for POST requests? Like so:

public static function get($url, $headers = array(), $data = array(), $options = array()) {
    return self::request($url, $headers, $data, self::GET, $options);
}

Then set up the request like the POST request in terms of parameters, placing the query inside the $data variable:

$request = Requests::get($url, $headers, $data, $options);

Am I missing something for how GET requests are handled? If so is there an example that includes the query string/data?

Thanks

The Host header has no port value when port is not 80

When the host has port, say http://www.abc.com:8888/path. The Host header for the request should be Host:www.abc.com:8888, but Requests will send Host:ww.abc.com instead.

Follow the RFC 2616, this is not correct. If the port value is not 80, the Host should contains the port. This affects some server library which follows the RFC.

The requested package rmccue/requests could not be found in any version, t here may be a typo in the package name.

anyone had the same error?

$ ../../../../php/php.exe composer.phar update
Loading composer repositories with package information
Updating dependencies
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested package rmccue/requests could not be found in any version, t
here may be a typo in the package name.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your min
imum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> f
or more details.

Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common
 problems.

Multine requests curl -d

Hi,

Is-it possible to use Requests to do a multiline get request, for example :

$ curl -XGET 'http://localhost:9200/twitter/tweet/_search?routing=kimchy' -d '{
    "query": {
        "filtered" : {
            "query" : {
                "query_string" : {
                    "query" : "some query string here"
                }
            },
            "filter" : {
                "term" : { "user" : "kimchy" }
            }
        }
    }
}
'

Thx for your help.

Create CLI example

As a proof-of-concept, Requests should include an example of how to use it via a CLI. What we need here is a tiny client that can make requests and do some common tasks.

• Send a GET request
• Display progress bar (to stderr, ala cURL)
• Enable streaming to file (-o)

Props to @rdlowrey for inspiration from Artax's Progress extension.

Add composer.json

add docs/usage-advanced.md

The document is missing!

Uncaught exceptions ?

Hi there

I'm currently playing with Requests and see if I can use it in YOURLS. I'm testing it under various cases and found one that doesn't work too well : when the transport is fsockopen and there's no SSL support, you end up with a fatal error if trying to perform a request on an https:// URL :

( ! ) Fatal error: Uncaught exception 'Requests_Exception' with message 'Unable to find the socket transport "ssl" - did you forget to enable it when you configured PHP?' in blah\Requests\Transport\fsockopen.php on line 97
( ! ) Requests_Exception: Unable to find the socket transport "ssl" - did you forget to enable it when you configured PHP? in blah\Requests\Transport\fsockopen.php on line 97

Not sure I understand everything correctly but it seems to me that no exception is caught, ever, hence whenever you're throwing a new one, it will die with a fatal error?

Composer

Hi,

Are there people using Requests with Composer ?

I use composer with monolog for example and have no issues, but requests won't work and keep telling me that it can't find the class (Requests), seems there is a problem with the path, but can't figure out which one....

I can see the autoload field is not set in the composer.json, I have tried various things, nothing works...

POST with basic auth broken with the fsockopen transport

First off, thanks for releasing this. Coming from Python, I was really glad to have a familiar library to use for the tiny bit of PHP I needed to write.

I stumbled on some incorrect line breaks in the headers produced by the fsockopen transport. I noticed the problem when using basic auth, but it will break if you give it any custom headers too.

The fix is simple: add \r\n at the end of this line, then remove the \r\n at the start of this line.

Thanks again.

Add PUT method support

Add ability to pass in a file pointer

How to post file data / upload file using requests? I can;t find any documentation on this.

Like in HttpRequest, we have a method called addPostFile()

How to Set cookies for use Reuqests?

have any example???

thanks,

Support for binary content

Requests doesn't allow send binary content in a POST operation.

In cURL transport class, on line 226 we have:

$data = http_build_query($data, null, '&');

In this line, the query is building by the function http_build_query, this query do a url encode that codify the character '@' to %40. Without this character the content isn't process by CURL like binary

Before build the query a validation must be done to know if the content could be a binary file.

[RFC] Split support for PHP into branches

Hey,

I have an idea, it's quite big because I would like to suggest you to:

• create new branch, called 1.7.x with code compatible with PHP 5.2.x
• remove compatibility for PHP 5.2.x from master branch (aka. i.e. 2.0)

My idea comes up mainly because support for PHP 5.2.x has been ended in 2010*.

(last was released at January 2011 but this was regression fix only)

Use higher level http library than curl?

I know that terminus is making some non-vanilla requests, but I wonder if it would make sense (assuming it's possible) to use a higher level library like HTTP_Request2 or Requests that would use alternative strategies if curl isn't on the system... and make for cleaner more readable connection code.

Support all HTTP verbs

The spesification for HTTP allows you to use any verb you want. Requests should allow me to do this.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

Edit: I love this library, I apologize if I came across as a bit grumpy :)

Add HTTP Digest Auth

It would be very nice to provide a built-in HTTP Digest Auth.

Fix composer package

Please:
define stable version via tags (or proper use of branches)
fix psr-0 or any class loading to be by default in autoloader (of composer).

Thanks now I will go play with this project Zend Http Client failed me miserably :)

Document the SSL options

These were introduced as part of #55. They should be documented in the prose section of the documentation, as they're fairly important changes, and a backwards compatibility change from 1.5.

Website ZIP outdated

I downloaded the zip from the website, it downloaded an old version, which was missing PUT and DELETE. Since those were added several months ago, the zip file linked form the website must be ancient.

If the code in github is always 'good', it might be good to link the zip button to the URL:
https://github.com/rmccue/Requests/zipball/master

If not, it might be about time to tag a new version and link to that.

Builds break randomly

Unfortunately, builds are a bit unreliable, since we use httpbin.org directly for a lot of testing. This also makes it hard to test when disconnected from the internet.

To fix this, we should pull down httpbin's code and use it directly on the testing machine.

Tag a version with Composer support

could you please tag a stable version for packagist / composer?
ATM there are only DEV versions available...

https://packagist.org/packages/rmccue/requests

ihost has private access

phpStorm complains at line 322 of Requests.php that ihost has private access in Requests_IRI

        $iri->host = Requests_IDNAEncoder::encode($iri->ihost);

Memory leak in cURL transport

I discovered a memory leak in the cURL transport. When doing multiple Requests::request() calls (for example in a loop) the memory usage keeps growing (in my case by 3–5MB every call).

I’ve narrowed it down to $this->fp not being closed in the cURL transport’s request() method. Now, this shouldn’t cause the memory leak, as a transport is instantiated in the scope of the Requests::request() method so it (and thus all its instance variables, including $this->fp) should be released when that method call completes.

I tried looking for a reference to the transport which is not released but can’t find any.

The temporary solution is to add curl_close($this->fp) just before the Requests_Transport_cURL::request() returns. Although it does fix the problem, it does not fix it at the source.

Before I fork your repo and add the temporary fix I thought I’d let you know so maybe you can find the real cause of this bug — since you’re more familiar with your code.

GET Request problem

Hey,

first of all thank you for your awesome php lib :)

But I think I've found a bug... Using a GET Request the header is empty, if I load the same URL in a Browser it displays the response correctly. I have the latest version, but I tested it also with 1.5, in both it isn't working. I debugged a little bit (I don't have much time) and the problem is between line number 320 and 323 (and the functions in them).
https://github.com/kurtextrem/ge.tt-PHP-API/blob/master/gett.class.php#L100 That's the line where it's called and because the header is empty this exception occurs "Missing header/body separator"

Hope you can help me or fix it.

Regards,
Jacob

Add ability to get raw HTTP data

At the moment, we parse the data and discard it. Some people might want this in their scripts, so we should make it available via Requests_Response

use of Requests with PHP 5.2.x is unclear.

Requests is ISC Licensed (similar to the new BSD license) and has no dependencies, except for PHP 5.2+.

a) can be read as "has no dependencies except for PHP 5.2+" or "(ISC Licensed and has no dependencies) except for PHP 5.2+".
b) what are those dependencies.

Autoloader fails after extensive usage

Hi,

I've been having this bug quite often lately and googling the error does not seem to shed any light on the matter.

I've created a wrapper of Requests to use on a particular API I'm using.

In my constructor I call Requests::register_autoloader() and then make regular calls using methods. Most times there is no problem, but in one particular script I'm getting an error from PHP complaining about too many open files.

Here's the errors I'm getting:

Warning: require_once(/.../vendor/rmccue/requests/library/Requests/Exception.php): failed to open stream: Too many open files in /.../vendor/rmccue/requests/library/Requests.php on line 115

PHP Fatal error:  require_once(): Failed opening required '/.../vendor/rmccue/requests/library/Requests/Exception.php' (include_path='.:/usr/local/Cellar/php54/5.4.7/lib/php') in /.../vendor/rmccue/requests/library/Requests.php on line 115

As a side question, why do you make use of you own autoloader instead of letting say composer handle the autoloading for you and use namespaces? Could this issue be solved if we used namespace autoloading?

I'll investigate further and if I find a fix I'll send you a pull request.

Thanks,
Alex

Set up PEAR server

cURL POST fields are added via CURLOPT_POSTFIELDS as an array, setting form to multipart/form-data

Requests_Transport_cURL::setup_handle() directly passes a PHP array to CURLOPT_POSTFIELDS when performing POST or PUT. According to http://php.net/manual/en/function.curl-setopt.php, this will implicitly set the request header Content-Type to a value of multipart/form-data. This is not always desired.

Would it be possible to add a feature whereby Requests encodes the POST array to a string when using cURL as the transport, perhaps via options? This would ensure that php-curl does not set the Content-Type header. The multipart form header causes problems in certain cases, and my workaround of converting POST array data into a string prior to calling Requests::request() is a bit sloppy.

Thanks.

Move cacert.pem to top-level dir

Currently, it's in library/Requests/Transport/, which is a bit strange, since you only expect to find PHP files there.

Context: wp-cli/wp-cli@24e296d

Disclose min requirements

Hi there

I couldn't find any info about minimum requirements (PHP 5.x I guess, but how many is the "x"?)
Might be worth it to mention it explicitly somewhere in the doc (or just point me to it if I failed at finding it :)

microsecond timeouts?

Hi

Correct me if I'm wrong but from reading request() phpdoc's it seems to me that you can only specify timeout in seconds? Is there any support for microseconds?

303 should change method to GET

As per RFC 2616, a 303 See Other should have the method changed to GET.

website borked

try it: http://requests.ryanmccue.info/

Handling timeouts?

The docs says that one of Request's features is "Connection Timeouts", but I don't see anything in the docs about that.

Any example code on how to, say, do a get request that gives up after 5 seconds?

Thanks!

Requests_Hooks not found

php file contains

require 'lib/Requests/library/Requests.php';
Requests::register_autoloader();

but receive this:

PHP Fatal error: Class 'Requests_Hooks' not found in /Users/lukeholder/Projects/Luke/lib/Requests/library/Requests.php on line 307

Add tests for SSL options

We should check that the SSL options work correctly by verifying with known correct and incorrect sites, as well as with a custom CA.

Release version 1.6

Issues remaining for 1.6:

• SSL certificates (security) in #63
• Sessions (to match cookies) in #62
• Small documentation issue in #33

The aim is to push the release out this week as soon as I have spare time. Also considering bumping to 2.0 given the size of the release, but we'll see.

Add ability to get raw header array

Requests_Response_Headers should be able to return a raw array for dumping purposes.

(This should also have the original case for the name, which we'll need another array for. Not sure if I'll implement this in a fancy way, since they are insensitive anyway, so we shouldn't be worried about multiple headers with different cases)

Parse cURL error messages for better exceptions

Our exceptions for cURL at the moment fall into a single bucket. Rather than having transport-specific exceptions, these should use a common set of names.

(In addition, we can probably improve the names and namespacing.)

Add DELETE method support

Request doen't work with cURL 7.15.5

The constant CURLOPT_PROTOCOLS doesn't exists in versions of cURL < 7.19.4

When a new transport object of cURL is created in line 71 the CURLOPT_PROTOCOLS defines the supported protocols, in this case only HTTP is allowed (it makes sense as we only work with HTTP).

The problem is the fact that CURLOPT_PROTOCOLS constant doesn't exists in versions of cURL < 7.19.4

A simple workaround can be:

if (version_compare($this->version, '7.19.4', '>=')) {
    curl_setopt($this->fp, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
}

Is a fact that we don't honor the restriction of use only HTTP protocols but we can use Requests library in version of cURL with this limitation. Not always we can upgrade the version of an extension in a production enviroment without messing existing code.

Add PATCH method support

kind of async support

I would like to propose an additional async like approach for Request. The client API would look something like this:

// trigger the requests
$future = Requests::request_async($requests, $options);

// do whatever PHP stuff you like
echo "lala";

// ...
// after you have done everything which is not related to the asyncRequest

 // wait for all curl requests which are not finished yet and 
// returns their results as request_multiple would do.
// also frees all remaining resources
$results = $future->getResult();

// process your result

to implement this kind of handling I would like to split Requests_Transport_cURL::request_multiple() in two smaller parts.

The first request_async part would start the requests (curl_init, curl_setopt, curl_multi_init, curl_multi_add_handle and kick of the requests using curl_multi_exec).

The second part $future->getResult() will run the loop over curl_multi_exec until everything is finished, cleanup using curl_multi_remove_handle & curl_multi_close.

this kind of request handling is AFAIK only possible using curl, therefore the default implementation would do the stuff synchrously (call request_mulitple).

I am willing to provide the implementation in case you give your 👍.

Nested parameters converted to strings

Given a request like this:

$params = array(
  'foo' => array('bar' => 'asdf')
);

Request::post($some_url, array(), $params);

This ends up posting over as 'foo=Array' instead of 'foo[bar]=asdf'. Right now the workaround it to pass it in as Request::post($some_url, array(), http_build_query($params));, but it seems like it should be detected and handled by the library.

cURL Connections Not Being Closed

There's a bug in the cURL transport that's causing curl_close to never be called, I added a debug line to process_response and confirmed it's never being called.

This is causing cURL requests to hang with CLOSE_WAIT, and for me, it ended up manifesting itself in a too many open files exception.

URL reformatting issue with Requests_IRI

When requesting an URL with a fragment, the reformatted URL can be different from the original one. The following example rewrites https://example.com/#foobar to https://example.com#foobar instead of returning the same URL.
As a consequence, curl fails to load the URL.

$url = "https://example.com/#foobar";
$iri = new Requests_IRI($url);
$url = $iri->uri;
var_dump($url);