AnyEvent::HTTP - simple but non-blocking HTTP/HTTPS client
use AnyEvent::HTTP;
http_get "http://www.nethype.de/", sub { print $_[1] };
# ... do something else here
This module is an AnyEvent user, you need to make sure that you use and run a supported event loop.
This module implements a simple, stateless and non-blocking HTTP client. It supports GET, POST and other request methods, cookies and more, all on a very low level. It can follow redirects supports proxies and automatically limits the number of connections to the values specified in the RFC.
It should generally be a "good client" that is enough for most HTTP tasks. Simple tasks should be simple, but complex tasks should still be possible as the user retains control over request and response headers.
The caller is responsible for authentication management, cookies (if the simplistic implementation in this module doesn't suffice), referer and other high-level protocol details for which this module offers only limited support.
Executes an HTTP-GET request. See the http_request function for details on additional parameters and the return value.
Executes an HTTP-HEAD request. See the http_request function for details on additional parameters and the return value.
Executes an HTTP-POST request with a request body of $body. See the
http_request function for details on additional parameters and the return
value.
Executes a HTTP request of type $method (e.g. GET, POST). The URL
must be an absolute http or https URL.
When called in void context, nothing is returned. In other contexts,
http_request returns a "cancellation guard" - you have to keep the
object at least alive until the callback get called. If the object gets
destroyed before the callbakc is called, the request will be cancelled.
The callback will be called with the response body data as first argument
(or undef if an error occured), and a hash-ref with response headers as
second argument.
All the headers in that hash are lowercased. In addition to the response
headers, the "pseudo-headers" (uppercase to avoid clashing with possible
response headers) HTTPVersion, Status and Reason contain the
three parts of the HTTP Status-Line of the same name.
The pseudo-header URL contains the actual URL (which can differ from
the requested URL when following redirects - for example, you might get
an error that your URL scheme is not supported even though your URL is a
valid http URL because it redirected to an ftp URL, in which case you can
look at the URL pseudo header).
The pseudo-header Redirect only exists when the request was a result
of an internal redirect. In that case it is an array reference with
the ($data, $headers) from the redirect response. Note that this
response could in turn be the result of a redirect itself, and $headers->{Redirect}[1]{Redirect} will then contain the original
response, and so on.
If the server sends a header multiple times, then their contents will be
joined together with a comma (,), as per the HTTP spec.
If an internal error occurs, such as not being able to resolve a hostname,
then $data will be undef, $headers->{Status} will be 59x
(usually 599) and the Reason pseudo-header will contain an error
message.
A typical callback might look like this:
sub {
my ($body, $hdr) = @_;
if ($hdr->{Status} =~ /^2/) {
... everything should be ok
} else {
print "error, $hdr->{Status} $hdr->{Reason}\n";
}
}
Additional parameters are key-value pairs, and are fully optional. They include:
Whether to recurse requests or not, e.g. on redirects, authentication retries and so on, and how often to do so.
The request headers to use. Currently, http_request may provide its
own Host:, Content-Length:, Connection: and Cookie: headers
and will provide defaults for User-Agent: and Referer: (this can be
suppressed by using undef for these headers in which case they won't be
sent at all).
The time-out to use for various stages - each connect attempt will reset the timeout, as will read or write activity, i.e. this is not an overall timeout.
Default timeout is 5 minutes.
Use the given http proxy for all requests. If not specified, then the
default proxy (as specified by $ENV{http_proxy}) is used.
$scheme must be either missing, http for HTTP or https for
HTTPS.
The request body, usually empty. Will be-sent as-is (future versions of this module might offer more options).
Passing this parameter enables (simplified) cookie-processing, loosely based on the original netscape specification.
The $hash_ref must be an (initially empty) hash reference which will
get updated automatically. It is possible to save the cookie_jar to
persistent storage with something like JSON or Storable, but this is not
recommended, as expiry times are currently being ignored.
Note that this cookie implementation is not of very high quality, nor
meant to be complete. If you want complete cookie management you have to
do that on your own. cookie_jar is meant as a quick fix to get some
cookie-using sites working. Cookies are a privacy disaster, do not use
them unless required to.
Specifies the AnyEvent::TLS context to be used for https connections. This
parameter follows the same rules as the tls_ctx parameter to
AnyEvent::Handle, but additionally, the two strings low or
high can be specified, which give you a predefined low-security (no
verification, highest compatibility) and high-security (CA and common-name
verification) TLS context.
The default for this option is low, which could be interpreted as "give
me the page, no matter what".
In rare cases you need to "tune" the socket before it is used to
connect (for exmaple, to bind it on a given IP address). This parameter
overrides the prepare callback passed to AnyEvent::Socket::tcp_connect
and behaves exactly the same way (e.g. it has to provide a
timeout). See the description for the $prepare_cb argument of
AnyEvent::Socket::tcp_connect for details.
When specified, this callback will be called with the header hash as soon as headers have been successfully received from the remote server (not on locally-generated errors).
It has to return either true (in which case AnyEvent::HTTP will continue),
or false, in which case AnyEvent::HTTP will cancel the download (and call
the finish callback with an error code of 598).
This callback is useful, among other things, to quickly reject unwanted
content, which, if it is supposed to be rare, can be faster than first
doing a HEAD request.
Example: cancel the request unless the content-type is "text/html".
on_header => sub {
$_[0]{"content-type"} =~ /^text\/html\s*(?:;|$)/
},
When specified, all body data will be passed to this callback instead of to the completion callback. The completion callback will get the empty string instead of the body data.
It has to return either true (in which case AnyEvent::HTTP will continue),
or false, in which case AnyEvent::HTTP will cancel the download (and call
the completion callback with an error code of 598).
This callback is useful when the data is too large to be held in memory (so the callback writes it to a file) or when only some information should be extracted, or when the body should be processed incrementally.
It is usually preferred over doing your own body handling via
want_body_handle, but in case of streaming APIs, where HTTP is
only used to create a connection, want_body_handle is the better
alternative, as it allows you to install your own event handler, reducing
resource usage.
When enabled (default is disabled), the behaviour of AnyEvent::HTTP
changes considerably: after parsing the headers, and instead of
downloading the body (if any), the completion callback will be
called. Instead of the $body argument containing the body data, the
callback will receive the AnyEvent::Handle object associated with the
connection. In error cases, undef will be passed. When there is no body
(e.g. status 304), the empty string will be passed.
The handle object might or might not be in TLS mode, might be connected to a proxy, be a persistent connection etc., and configured in unspecified ways. The user is responsible for this handle (it will not be used by this module anymore).
This is useful with some push-type services, where, after the initial headers, an interactive protocol is used (typical example would be the push-style twitter API which starts a JSON/XML stream).
If you think you need this, first have a look at on_body, to see if
that doesn't solve your problem in a better way.
Example: make a simple HTTP GET request for http://www.nethype.de/
http_request GET => "http://www.nethype.de/", sub {
my ($body, $hdr) = @_;
print "$body\n";
};
Example: make a HTTP HEAD request on https://www.google.com/, use a timeout of 30 seconds.
http_request
GET => "https://www.google.com",
timeout => 30,
sub {
my ($body, $hdr) = @_;
use Data::Dumper;
print Dumper $hdr;
}
;
Example: make another simple HTTP GET request, but immediately try to cancel it.
my $request = http_request GET => "http://www.nethype.de/", sub {
my ($body, $hdr) = @_;
print "$body\n";
};
undef $request;
AnyEvent::HTTP uses the AnyEvent::Socket::tcp_connect function for
the actual connection, which in turn uses AnyEvent::DNS to resolve
hostnames. The latter is a simple stub resolver and does no caching
on its own. If you want DNS caching, you currently have to provide
your own default resolver (by storing a suitable resolver object in
$AnyEvent::DNS::RESOLVER).
Sets the default proxy server to use. The proxy-url must begin with a
string of the form http://host:port (optionally https:...), croaks
otherwise.
To clear an already-set proxy, use undef.
The default value for the recurse request parameter (default: 10).
The default value for the User-Agent header (the default is
Mozilla/5.0 (compatible; U; AnyEvent-HTTP/$VERSION; +http://software.schmorp.de/pkg/AnyEvent)).
The maximum number of concurrent connections to the same host (identified by the hostname). If the limit is exceeded, then the additional requests are queued until previous connections are closed.
The default value for this is 4, and it is highly advisable to not
increase it.
The number of active connections. This is not the number of currently running requests, but the number of currently open and non-idle TCP connections. This number of can be useful for load-leveling.
AnyEvent.
Marc Lehmann <schmorp@schmorp.de> http://home.schmorp.de/
With many thanks to Дмитрий Шалашов, who provided countless testcases and bugreports.