Coder Social home page Coder Social logo

www's Introduction

Logo | TestTest-create and publish Docker images

NAME

WWW - No-nonsense, simple HTTPS client with JSON decoder

SYNOPSIS

use WWW;

# Just GET content (will return Failure on failure):
say get 'https://httpbin.org/get?foo=42&bar=x', :SomeHeader<Value>;

# GET and decode received data as JSON:
say jget('https://httpbin.org/get?foo=42&bar=x')<args><foo>;

# POST content (query args are OK; pass form as named args)
say post 'https://httpbin.org/post?foo=42&bar=x', :some<form>, :42args;

# And if you need headers, pass them inside a positional Hash:
say post 'https://httpbin.org/post?foo=42&bar=x', %(:Some<Custom-Header>),
    :some<form>, :42args;

# Same POST as above + decode response as JSON
say jpost('https://httpbin.org/post', :some<form-arg>)<form><some>;

# Also can post() or jpost() POST body directly as Str:D; headers passed as named args:
say jpost(
    "http://httpbin.org/post",
    to-json({:42a, :foo<meows>}),
    :Authorization<Zofmeister>
).<json><foo>;

Import more HTTP methods using :extras

use WWW :extras;

say jdelete 'https://httpbin.org/delete';
say jput    'https://httpbin.org/put';

DESCRIPTION

Exports a handful of routines to fetch data from online resources using HTTP verbs and optionally decode the responses as JSON.

The module will set the User-Agent header to Rakudo WWW, unless you specify that header.

INSTALLATION

On some operating systems you'll need libssl installed:

sudo apt-get install libssl-dev

Then just install the module with the module manager:

zef install WWW

TESTING

To run the full test suite, set ONLINE_TESTING environmental variable to 1

ONLINE_TESTING=1 zef install WWW

EXPORTED ROUTINES

:DEFAULT export tag

These routines get exported by default:

get

sub get($url where URI:D|Str:D, *%headers --> Str:D);

say get 'https://httpbin.org/get?foo=42&bar=x';

Takes either a Str or a URI. Returns Failure if request fails or does not return a successful HTTP code. Returns Str with the data on success. Takes headers as named arguments.

jget

say jget 'https://httpbin.org/get?foo=42&bar=x';

Same as get() except will also decode the response as JSON and return resultant data structure.

post

multi post($url where URI:D|Str:D, *%form --> Str:D);
multi post($url where URI:D|Str:D, %headers, *%form --> Str:D);
multi post($url where URI:D|Str:D, Str:D $form-body, *%headers --> Str:D);

say post 'https://httpbin.org/post?meow=moo', :72foo, :bar<>;
say post 'https://httpbin.org/post?meow=moo',
    %(Content-type => 'application/json'), :72foo, :bar<>;

Takes either a Str or a URI, followed by an optional Hash with HTTP headers to send. Form POST parameters can be included as named arguments. It's fine to also include query arguments in the URL itself. Returns Failure if request fails or does not return a successful HTTP code. Returns Str with the data on success.

To send POST body directly, pass it as Str:D positional arg. In this calling form, the headers are sent as named args.

jpost

multi jpost($url where URI:D|Str:D, *%form);
multi jpost($url where URI:D|Str:D, %headers, *%form);
multi jpost($url where URI:D|Str:D, Str:D $form-body, *%headers);

say jpost 'https://httpbin.org/post?meow=moo', :72foo, :bar<>;
say jpost 'https://httpbin.org/post?meow=moo',
    %(Content-type => 'application/json'), :72foo, :bar<>;

Same as post() except will also decode the response as JSON and return resultant data structure.

head

say head 'https://httpbin.org/get?foo=42&bar=x';

Same as get, except it does not actually download the content, just the head.

:extras Export Tag

These routines get exported in addition to the :DEFAULT exports, when :extras export tag is requested:

use WWW :extras;

put

multi put($url where URI:D|Str:D, *%form --> Str:D);
multi put($url where URI:D|Str:D, %headers, *%form --> Str:D);
multi put($url where URI:D|Str:D, Str:D $form-body, *%headers --> Str:D);

say put 'https://httpbin.org/put?meow=moo', :72foo, :bar<>;
say put 'https://httpbin.org/put?meow=moo',
    %(Content-type => 'application/json'), :72foo, :bar<>;

Takes either a Str or a URI, followed by an optional Hash with HTTP headers to send. Form PUT parameters can be included as named arguments. It's fine to also include query arguments in the URL itself. Returns Failure if request fails or does not return a successful HTTP code. Returns Str with the data on success.

To send PUT body directly, pass it as Str:D positional arg. In this calling form, the headers are sent as named args.

jput

multi jput($url where URI:D|Str:D, *%form --> Str:D);
multi jput($url where URI:D|Str:D, %headers, *%form --> Str:D);
multi jput($url where URI:D|Str:D, Str:D $form-body, *%headers --> Str:D);

say jput 'https://httpbin.org/put?meow=moo', :72foo, :bar<>;
say jput 'https://httpbin.org/put?meow=moo',
    %(Content-type => 'application/json'), :72foo, :bar<>;

Same as put() except will also decode the response as JSON and return resultant data structure.

delete

sub delete($url where URI:D|Str:D, *%headers --> Str:D);

say delete 'https://httpbin.org/get?foo=42&bar=x';

Performs HTTP DELETE request. Takes either a Str or a URI. Returns Failure if request fails or does not return a successful HTTP code. Returns Str with the data on success; if response for a 204 No Content, returns an empty string. Takes headers as named arguments.

jdelete

say jdelete 'https://httpbin.org/get?foo=42&bar=x';

Same as delete() except will also decode the response as JSON and return resultant data structure.

You probably want to use delete() instead, as DELETE requests can get return no content, causing JSON parse failures.

LIMITATIONS

Due to nuances of upstream code, currently any non-RFC-conformant URL will be rejected; you have to ensure it's proper manually. Patches welcome.

SEE ALSO


REPOSITORY

Fork this module on GitHub: https://github.com/raku-community-modules/WWW

BUGS

To report bugs or request features, please use https://github.com/raku-community-modules/WWW/issues

ORIGINAL AUTHOR

Zoffix Znet (http://perl6.party/)

Now maintained by the Raku community as part of the Raku community modules

LICENSE

You can use and distribute this module under the terms of the The Artistic License 2.0. See the LICENSE file included in this distribution for complete details.

The META6.json file of this distribution may be distributed and modified without restrictions or attribution.

www's People

Contributors

altai-man avatar jj avatar lizmat avatar samcv avatar wictory avatar zoffixznet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

www's Issues

Create Dockerfile for testing

Since testing takes like 10 minutes due to the long installation of modules,
it's probably the best to create a custom Dockerfile for testing.

This is needed too since it uses openssl. No big deal anyway.

Installing via Zef fails due to test failures.

I am using rakudo version 2017.03 running 'zef install WWW' on ubuntu 16.10

I am getting the following output.

t/01-basic.t ..... ok 
t/02-is-tests.t .. ok  
All tests successful.
Files=2, Tests=46,  0 wallclock secs
Result: PASS
===> Testing [OK] for HTTP::Status
===> Testing: File::Directory::Tree:auth('labster')
t/basic.t .. ok 
All tests successful.
Files=1, Tests=7,  0 wallclock secs
Result: PASS
===> Testing [OK] for File::Directory::Tree:auth('labster')
===> Testing: File::Temp
t/01-load.t ....... ok 
t/02-stress_gc.t .. ok 
t/03-tempfile.t ... ok  
All tests successful.
Files=3, Tests=18,  1 wallclock secs
Result: PASS
===> Testing [OK] for File::Temp
===> Testing: DateTime::Parse:ver('0.0.1')
t/01-basic.t .. ok  
All tests successful.
Files=1, Tests=11,  0 wallclock secs
Result: PASS
===> Testing [OK] for DateTime::Parse:ver('0.0.1')
===> Testing: Encode:ver('0.0.2'):auth('github:sergot')
t/01-basic.t ........ ok 
t/02-latin2.t ....... ok 
t/03-utf8.t ......... ok 
t/04-latin1.t ....... ok 
t/05-ascii.t ........ ok 
t/06-windows1252.t .. ok 
t/07-windows1251.t .. ok 
All tests successful.
Files=7, Tests=9,  1 wallclock secs
Result: PASS
===> Testing [OK] for Encode:ver('0.0.2'):auth('github:sergot')
===> Testing: MIME::Base64:ver('1.2'):auth('github:retupmoca')
t/basic.t ................. ok  
t/binary-and-long-line.t .. ok  
t/oneline.t ............... ok 
t/rfc4648-test-vector.t ... ok  
All tests successful.
Files=4, Tests=47,  1 wallclock secs
Result: PASS
===> Testing [OK] for MIME::Base64:ver('1.2'):auth('github:retupmoca')
===> Testing: URI:ver('0.1.3')
t/01.t ................... ok  
t/escape.t ............... ok  
t/november-urlencoded.t .. ok  
t/require.t .............. ok 
t/rfc-3986-examples.t .... ok  
t/utf8-c8.t .............. ok 
All tests successful.
Files=6, Tests=96,  2 wallclock secs
Result: PASS
===> Testing [OK] for URI:ver('0.1.3')
===> Testing: IO::Capture::Simple
t/stderr.t .................... ok 
t/stdout.t .................... ok 
t/test-io-capture/01-basic.t .. ok 
All tests successful.
Files=3, Tests=7,  0 wallclock secs
Result: PASS
===> Testing [OK] for IO::Capture::Simple
===> Testing: Test::Util::ServerPort:ver('0.0.1'):auth('github:jonathanstowe')
t/010-basic.t .. ok 
All tests successful.
Files=1, Tests=8,  0 wallclock secs
Result: PASS
===> Testing [OK] for Test::Util::ServerPort:ver('0.0.1'):auth('github:jonathanstowe')
===> Testing: HTTP::UserAgent:ver('1.1.30'):auth('github:sergot')
t/010-headers.t ............ ok  
t/020-message.t ............ ok  
t/030-cookies.t ............ ok  
t/040-request.t ............ ok  
t/050-response.t ........... ok  
t/060-ua-common.t .......... ok 
t/070-ua-simple.t .......... ok 
t/080-ua.t .................1/10# NETWORK_TESTING was not set
t/080-ua.t ................. ok  

# Failed test 'and it isn't successful'
# at t/082-exceptions.t line 12

# Failed test 'and a 404'
# at t/082-exceptions.t line 13
t/082-exceptions.t .........1/?# expected: '404'
#      got: '200'

    # Failed test 'code dies'
    # at t/082-exceptions.t line 17
    # Looks like you failed 1 test of 4

# Failed test 'did we throws-like HTTP::UserAgent::X::HTTP::Response?'
# at t/082-exceptions.t line 17
# Looks like you failed 3 tests of 4
t/082-exceptions.t ......... Failed 3/4 subtests 
t/085-auth.t ............... ok 
t/090-ua-ssl.t ............. ok 
t/100-redirect-ssl.t ....... ok 
t/110-redirect-cookies.t ... ok 
# NETWORK_TESTING was not set
t/150-issue-64.t ........... ok 
t/160-issue-67.t ........... ok 
t/170-request-common.t ..... ok 
t/180-mediatype.t .......... ok 
t/190-issue-116.t .......... ok 
# NETWORK_TESTING was not set
t/200-w3-test-encodings.t .. ok 
t/210-content-encoding.t ... ok 
t/220-binary-content.t ..... ok 
t/230-binary-request.t ..... ok 
t/250-issue-144.t .......... ok 
t/260-no-proxy.t ........... ok 

Test Summary Report
-------------------
t/082-exceptions.t  (Wstat: 0 Tests: 4 Failed: 3)
  Failed tests:  2 3 4
Files=24, Tests=209,  13 wallclock secs
Result: FAILED
===> Testing [FAIL]: HTTP::UserAgent:ver('1.1.30'):auth('github:sergot')
Aborting due to test failure: HTTP::UserAgent:ver('1.1.30'):auth('github:sergot') (use --force to override)
  in code  at /home/maxim/.perl6/sources/9AC28480023ECA9DD4DC381AF0ACBFB147CF1D69 (Zef::Client) line 346
  in method test at /home/maxim/.perl6/sources/9AC28480023ECA9DD4DC381AF0ACBFB147CF1D69 (Zef::Client) line 325
  in code  at /home/maxim/.perl6/sources/9AC28480023ECA9DD4DC381AF0ACBFB147CF1D69 (Zef::Client) line 497
  in sub  at /home/maxim/.perl6/sources/9AC28480023ECA9DD4DC381AF0ACBFB147CF1D69 (Zef::Client) line 494
  in method install at /home/maxim/.perl6/sources/9AC28480023ECA9DD4DC381AF0ACBFB147CF1D69 (Zef::Client) line 600
  in sub MAIN at /home/maxim/.perl6/sources/EC05552BF10F0D7777355946A3534A3099A0726F (Zef::CLI) line 139
  in block <unit> at /home/maxim/.perl6/resources/F32C337878899C8DC1C842AA735BB40EBC9F70E3 line 1


Support for other http methods (puts and head)

Hi, would it be possible to add support for put and head http methods to this library?
This would allow it to be easily used when people are writing api libraries.

I'd like to take a stab this but I'm unsure where to start, not having written much perl6 before.

Fix Travis setup

Need to fix Test::When's tests to set all other env vars to false

Can't get it to work

I want to post a JSON. I have tried:

 jpost("https://www.googleapis.com/urlshortener/v1/url?key=$api-key",
			     to-json({:longUrl<$this-url>}),
			     :Content-Type<application/json>,

and several other possible ways. I get 400 response; I'm trying to tap https://developers.google.com/url-shortener/v1/getting_started
I'm getting "Bad request" all the time, indicating the longUrl parameter has not been set correctly.

Failure to install with zef

$ zef install WWW
===> Searching for: WWW
===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
===> Fetching [FAIL]: WWW:ver<1.005004>:authgithub:perl6-community-modules from https://github.com/perl6-community-modules/WWW.git
Aborting due to fetch failure: WWW:ver<1.005004>:authgithub:perl6-community-modules (use --force-fetch to override)

macOS Catalina 10.15.7
Welcome to 𝐑𝐚𝐤𝐮𝐝𝐨™ v2020.10.
Implementing the 𝐑𝐚𝐤𝐮™ programming language v6.d.
Built on MoarVM version 2020.10.

Incorrecly encoded query args seem to be just ignored

When making the request below, using geth's dev util script, the query would come in as just chan= if instead of encoded %23 an unencoded # was used:

constant URL = 'http://localhost:8888/?chan=%23perl6'; 
say "Response is: " ~ post URL, :X-GITHUB-EVENT<push>, get-req;

Feature request: Basic HTTP authentication

I've been trying to use this module for HTTP requests but I can't get past a 401 Unauthorized error for websites requiring basic HTTP authentication.

Are there any plans to add this functionality? Or am I doing something wrong?

This is irremediably out of sync with HTTP::UserAgent

More tests are failing (like #17), but in this case it's a syntax error which indicates that it's using the old signature for HTTP::UserAgent.
Also, HTTP::UserAgent is not qualified, while HTTP::Response is. That's probably not good.

different machines show different nesting levels for json output

I am running this script:

#!/usr/bin/env raku
use v6;
use WWW;

my @allRepos=jget qq!https://api.github.com/users/raku-community-modules/repos!;

say @allRepos.elems;

on two machines on the same network, running the same version of Raku:

$ raku -v
Welcome to 𝐑𝐚𝐤𝐮𝐝𝐨™ v2021.06-21-gdba9f56af.
Implementing the 𝐑𝐚𝐤𝐮™ programming language v6.d.
Built on MoarVM version 2021.06-6-g6977d0591.

and the same version of WWW, namely 1.005006. Furthermore, I have checked that

curl https://api.github.com/users/raku-community-modules/repos 

returns identical output on both machines (the file is here; I ran that curl command on both and diff-ed the outputs; they're identical).

Nevertheless, on one machine the Raku script provided above returns 1, and on the other it returns 30. For the first machine, I have to furter unwrap the output to @allRepos.[0] to get to the 30 repos.

So there's an extra level of nesting on just one machine, but I don't understand how this can be, given that everything seems to be identical.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.