Coder Social home page Coder Social logo

http-request-fromcurl's Issues

Test suite fails: Accept-Encoding does not exist

Seen on CentOS7 smokers:

"my" variable $res masks earlier declaration in same scope at t/TestWgetIdentity.pm line 177.
Missing argument in sprintf at t/TestWgetIdentity.pm line 188.
# Testing with wget version '1.14'

#   Failed test '-O - --debug --cache $url --header X-Test: test (Logs)'
#   at t/TestWgetIdentity.pm line 255.
#     Structures begin differing at:
#          $got->{Accept-Encoding} = undef
#     $expected->{Accept-Encoding} = Does not exist
# $VAR1 = [
#           {
#             "User-Agent" => "Wget/1.14",
#             "X-Test" => "test",
#             "Accept" => "*/*",
#             "Connection" => "Keep-Alive",
#             "Accept-Encoding" => undef,
#             "Host" => "[::1]:46149"
#           },
#           {
#             "Connection" => "Keep-Alive",
#             "User-Agent" => "Wget/1.14",
#             "X-Test" => "test",
#             "Accept" => "*/*",
#             "Host" => "[::1]:46149"
#           }
#         ];

#   Failed test 'We create (almost) the same headers with LWP'
#   at t/TestWgetIdentity.pm line 177.
#     Structures begin differing at:
#          $got->[3] = 'Accept-Encoding: identity'
#     $expected->[3] = 'Host: [::1]:46149'
... etc ...

improved translation of -F curl flag

Trying the following URL on https://corion.net/curl2lwp.psgi gives us some encoded form data

curl -X POST -F field=value -F name=Corion "https://httpbin.org/post" -H  "accept: application/json"

The following "translation" might be better:

use strict;
use warnings;

use LWP::UserAgent ();
use HTTP::Request::Common qw(POST);

my $ua = LWP::UserAgent->new(timeout => 10);
my $url = 'https://httpbin.org/post';
my %content = (
    field => 'value',
    name => 'Corion',
);

my $request = POST $url, 'Content-Type' => 'form-data', Content => \%content;
my $response = $ua->request($request);
print $response->decoded_content;

Tests started failing, maybe srand is missing?

Last night my smoker sent the 26 PASS reports listed below and after that all following reports were FAILs. I guess there should be some srand somewhere. The manpage of File::Temp insists that this is necessary in forking environments. IIUC, the tests are forking? Here is the link to the paragraph in the File::Temp manpage: https://metacpan.org/module/File::Temp/source#L2542-2546

Just for the record: my /tmp/ directory has at the moment about 1500 files and directories matching m!^/tmp/\w{10}$!; so if you cannot reproduce my findings, please be more patient.

Here's a link to a sample fail reports: http://www.cpantesters.org/cpan/report/23dd792a-709b-11ee-98af-ea102c1f115a

137511985 | 5.3.0-2-amd64 | x86_64-linux-ld | 5.28.2 | linux | Andreas J. Koenig | 2023-10-22 05:09
137511707 | 5.16.0-5-amd64 | x86_64-linux-thread-multi | 5.28.3 | linux | Andreas J. Koenig | 2023-10-22 04:52
137511669 | 5.16.0-6-amd64 | x86_64-linux-thread-multi | 5.28.3 | linux | Andreas J. Koenig | 2023-10-22 04:50
137511631 | 5.16.0-5-amd64 | x86_64-linux-thread-multi | 5.30.0 | linux | Andreas J. Koenig | 2023-10-22 04:48
137511585 | 5.17.0-1-amd64 | x86_64-linux-thread-multi | 5.30.1 | linux | Andreas J. Koenig | 2023-10-22 04:45
137511535 | 5.16.0-5-amd64 | x86_64-linux-ld | 5.30.2 | linux | Andreas J. Koenig | 2023-10-22 04:42
137511497 | 5.10.0-14-amd64 | x86_64-linux | 5.30.3 | linux | Andreas J. Koenig | 2023-10-22 04:40
137511467 | 5.15.0-2-amd64 | x86_64-linux-ld | 5.32.1 | linux | Andreas J. Koenig | 2023-10-22 04:38
137511442 | 5.16.0-6-amd64 | x86_64-linux-ld | 5.32.1 | linux | Andreas J. Koenig | 2023-10-22 04:35
137511354 | 6.1.0-7-amd64 | x86_64-linux-thread-multi | 5.32.1 | linux | Andreas J. Koenig | 2023-10-22 04:29
137511239 | 5.16.0-6-amd64 | x86_64-linux-thread-multi | 5.34.0 | linux | Andreas J. Koenig | 2023-10-22 04:22
137511212 | 5.17.0-1-amd64 | x86_64-linux-multi-ld | 5.34.1 | linux | Andreas J. Koenig | 2023-10-22 04:21
137511171 | 5.10.0-14-amd64 | x86_64-linux-thread-multi-quadmath | 5.36.0 | linux | Andreas J. Koenig | 2023-10-22 04:18
137511134 | 5.10.0-14-amd64 | x86_64-linux-thread-multi-ld | 5.36.0 | linux | Andreas J. Koenig | 2023-10-22 04:16
137511108 | 5.10.0-14-amd64 | x86_64-linux | 5.36.0 | linux | Andreas J. Koenig | 2023-10-22 04:15
137511007 | 6.1.0-7-amd64 | x86_64-linux-thread-multi-ld | 5.36.1 | linux | Andreas J. Koenig | 2023-10-22 04:09
137510951 | 6.1.0-7-amd64 | x86_64-linux-ld | 5.37.11 | linux | Andreas J. Koenig | 2023-10-22 04:05
137510916 | 6.1.0-7-amd64 | x86_64-linux-thread-multi-ld | 5.37.11 | linux | Andreas J. Koenig | 2023-10-22 04:03
137510797 | 6.1.0-7-amd64 | x86_64-linux-thread-multi | 5.37.11 | linux | Andreas J. Koenig | 2023-10-22 03:56
137510753 | 6.1.0-7-amd64 | x86_64-linux-thread-multi | 5.39.1 | linux | Andreas J. Koenig | 2023-10-22 03:53
137510687 | 6.1.0-7-amd64 | x86_64-linux-quadmath | 5.39.2 | linux | Andreas J. Koenig | 2023-10-22 03:47
137510629 | 6.1.0-7-amd64 | x86_64-linux-multi | 5.39.2 | linux | Andreas J. Koenig | 2023-10-22 03:42
137510479 | 6.1.0-7-amd64 | x86_64-linux-thread-multi-ld | 5.39.2 | linux | Andreas J. Koenig | 2023-10-22 03:31
137510396 | 6.1.0-7-amd64 | x86_64-linux-quadmath | 5.39.3 | linux | Andreas J. Koenig | 2023-10-22 03:26
137510294 | 6.1.0-7-amd64 | x86_64-linux-thread-multi-ld | 5.39.3 | linux | Andreas J. Koenig | 2023-10-22 03:19
137510191 | 6.1.0-7-amd64 | x86_64-linux-thread-multi | 5.39.3 | linux | Andreas J. Koenig | 2023-10-22 03:12

Test suite fails: content-length mismatch

On some systems a content-length check fails:

#   Failed test '--verbose -g -s --data-urlencode @$tempfile $url (reconstructed)'
#   at t/TestCurlIdentity.pm line 425.
#     Structures begin differing at:
#          $got->{headers}{Content-Length} = '34'
#     $expected->{headers}{Content-Length} = '26'
# Original command:
# $VAR1 = [
#           "--verbose",
#           "-g",
#           "-s",
#           "--data-urlencode",
#           "\@\$tempfile",
#           "\$url"
#         ];
# Original request:
# $VAR1 = {
#           "method" => "POST",
#           "headers" => {
#                          "User-Agent" => "curl/7.79.1",
#                          "Host" => "[::1]:30052",
#                          "Content-Length" => 26,
#                          "Content-Type" => "application/x-www-form-urlencoded",
#                          "Accept" => "*/*"
#                        },
#           "path" => "/",
#           "protocol" => "HTTP/1.1"
#         };
# Reconstructed command:
# $VAR1 = [
#           "--verbose",
#           "--silent",
#           "--request",
#           "POST",
#           "--header",
#           "Content-Length: 34",
#           "--header",
#           "Content-Type: application/x-www-form-urlencoded",
#           "--user-agent",
#           "curl/7.79.1",
#           "--data-raw",
#           "This%20is%20a%20test%0AMore%20test",
#           "http://[::1]:30052/"
#         ];
# Reconstructed request:
# $VAR1 = {
#           "protocol" => "HTTP/1.1",
#           "headers" => {
#                          "Accept" => "*/*",
#                          "Content-Type" => "application/x-www-form-urlencoded",
#                          "Host" => "[::1]:30052",
#                          "User-Agent" => "curl/7.79.1",
#                          "Content-Length" => 34
#                        },
#           "path" => "/",
#           "method" => "POST"
#         };

#   Failed test '--verbose -g -s --data-urlencode @$tempfile $url'
#   at t/TestCurlIdentity.pm line 276.
#     Structures begin differing at:
#          $got->{Content-Length} = '34'
#     $expected->{Content-Length} = '26'
# $VAR1 = [
#           {
#             "Content-Length" => 34,
#             "User-Agent" => "curl/7.79.1",
#             "Host" => "[::1]:30052",
#             "Accept" => "*/*",
#             "Content-Type" => "application/x-www-form-urlencoded"
#           },
#           {
#             "Host" => "[::1]:30052",
#             "User-Agent" => "curl/7.79.1",
#             "Accept" => "*/*",
#             "Content-Type" => "application/x-www-form-urlencoded",
#             "Content-Length" => 26
#           }
#         ];

#   Failed test 'We create (almost) the same headers with LWP'
#   at t/TestCurlIdentity.pm line 197.
#     Structures begin differing at:
#          $got->[5] = 'Content-Length: 34'
#     $expected->[5] = 'Content-Length: 26'
# Expected:
# Request:
# POST / HTTP/1.1
# Accept: */*
# Host: [::1]:30052
# User-Agent: curl/7.79.1
# Content-Length: 26
# Content-Type: application/x-www-form-urlencoded
# 
# This+is+a+test%0AMore+test
# 
# Got:
# Request:
# POST / HTTP/1.1
# Accept: */*
# Host: [::1]:30052
# User-Agent: curl/7.79.1
# Content-Length: 34
# Content-Type: application/x-www-form-urlencoded
# 
# This%20is%20a%20test%0AMore%20test
# 
#     use strict;
#      use LWP::UserAgent;
# 
#     my $ua = LWP::UserAgent->new('send_te' => '0');
#     my $r = HTTP::Request->new(
#         'POST' => 'http://[::1]:30052/',
#         [
#             'Accept' => '*/*',
#             'Host' => '[::1]:30052',
#             'User-Agent' => 'curl/7.79.1',
#             'Content-Length' => '34',
#             'Content-Type' => 'application/x-www-form-urlencoded'
#         ],
#         "This\x2520is\x2520a\x2520test\x250AMore\x2520test"
#     );
#     my $res = $ua->request( $r,  );
# 

#   Failed test 'We create (almost) the same headers with HTTP::Tiny'
#   at t/TestCurlIdentity.pm line 197.
#     Structures begin differing at:
#          $got->[4] = 'Content-Length: 34'
#     $expected->[4] = 'Content-Length: 26'
# Expected:
# Request:
# POST / HTTP/1.1
# Accept: */*
# User-Agent: curl/7.79.1
# Content-Length: 26
# Content-Type: application/x-www-form-urlencoded
# 
# This+is+a+test%0AMore+test
# 
# Got:
# Request:
# POST / HTTP/1.1
# Accept: */*
# User-Agent: curl/7.79.1
# Content-Length: 34
# Content-Type: application/x-www-form-urlencoded
# 
# This%20is%20a%20test%0AMore%20test
# 
#     use strict;
#      use HTTP::Tiny;
# 
#     my $ua = HTTP::Tiny->new('verify_SSL' => '1');
#     my $res = $ua->request(
#         'POST' => 'http://[::1]:30052/',
#         {
#           headers => {
#             'Content-Length' => '34',
#             'User-Agent' => 'curl/7.79.1',
#             'Accept' => '*/*',
#             'Content-Type' => 'application/x-www-form-urlencoded'
#           },
#           content =>  "This\x2520is\x2520a\x2520test\x250AMore\x2520test"
#         },
#     );
# 
# Looks like you failed 4 tests of 40.
t/curl-identity-data.t ....... 
Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/40 subtests

Test suite fails: user-agent mismatch

On almost all of my linux & freebsd smokers I see test failures like this:

#   Failed test '-O - --debug --no-cache $url --header X-Test: test (reconstructed)'
#   at t/TestWGetIdentity.pm line 390.
#     Structures begin differing at:
#          $got->{headers}{User-Agent} = 'Wget/1.18'
#     $expected->{headers}{User-Agent} = 'Wget/1.18 (freebsd10.3)'
...

Help! 

$ curl2perl -?
Unknown option: ?
$ curl2perl --help
Unknown option: help
$ curl2perl -help
Unknown option: h

Test suite fails: Cache-Control without must-revalidate

On some systems the test suite fails like this:

#   Failed test '-O - --debug --no-cache $url --header X-Test: test (reconstructed)'
#   at t/TestWGetIdentity.pm line 390.
#     Structures begin differing at:
#          $got->{headers}{Cache-Control} = 'no-cache'
#     $expected->{headers}{Cache-Control} = 'no-cache, must-revalidate'
# Original command:
# $VAR1 = [
#           "-O",
#           "-",
#           "--debug",
#           "--no-cache",
#           "\$url",
#           "--header",
#           "X-Test: test"
#         ];
# Original request:
# $VAR1 = {
#           "method" => "GET",
#           "protocol" => "HTTP/1.1",
#           "path" => "/",
#           "headers" => {
#                          "Cache-Control" => "no-cache, must-revalidate",
#                          "Accept" => "*/*",
#                          "Host" => "[::1]:54342",
#                          "User-Agent" => "Wget/1.14 (linux-gnu)",
#                          "X-Test" => "test",
#                          "Connection" => "Keep-Alive",
#                          "Pragma" => "no-cache"
#                        }
#         };
# Reconstructed command:
# $VAR1 = [
#           "--debug",
#           "-O",
#           "-",
#           "--header",
#           "Cache-Control: no-cache",
#           "--header",
#           "Pragma: no-cache",
#           "--user-agent",
#           "Wget/1.14",
#           "--header",
#           "X-Test: test",
#           "http://[::1]:54342/"
#         ];
# Reconstructed request:
# $VAR1 = {
#           "method" => "GET",
#           "protocol" => "HTTP/1.1",
#           "path" => "/",
#           "headers" => {
#                          "Pragma" => "no-cache",
#                          "X-Test" => "test",
#                          "Connection" => "Keep-Alive",
#                          "Host" => "[::1]:54342",
#                          "User-Agent" => "Wget/1.14",
#                          "Cache-Control" => "no-cache",
#                          "Accept" => "*/*"
#                        }
#         };

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.