httperf / httperf Goto Github PK

httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
httperf: maximum number of open descriptors = 1024

httperf --client=0/1 --server=localhost --port=9081 --uri=/bid/123 --send-buffer=4096 --recv-buffer=16384 --add-header='Content-Type: application/json\n' --method=POST --wsesslog=1000,100.000,http_perf.txt
httperf: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE
httperf: did not recognize arg '"ssp1-27009141-1485848975137","at":' in http_perf.txt

I have tried escaping " with \  , but it did not help either .
Please advise .

Using httperf with --hog option can prevent it binding sockets because the 
socket family is not set.

$ strace -tt httperf --hog
...
00:29:42.668802 bind(3, {sa_family=AF_UNSPEC, 
sa_data="PY\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family 
not supported by protocol)
00:29:42.668839 bind(3, {sa_family=AF_UNSPEC, 
sa_data="PZ\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family 
not supported by protocol)
00:29:42.668877 bind(3, {sa_family=AF_UNSPEC, 
sa_data="P[\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family 
not supported by protocol)
00:29:42.668914 bind(3, {sa_family=AF_UNSPEC, 
sa_data="P\\\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family 
not supported by protocol)
00:29:42.668950 bind(3, {sa_family=AF_UNSPEC, 
sa_data="P]\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family 
not supported by protocol)
00:29:42.668988 bind(3, {sa_family=AF_UNSPEC, 
sa_data="P^\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family 
not supported by protocol)
00:29:42.669025 bind(3, {sa_family=AF_UNSPEC, 
sa_data="P_\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family 
not supported by protocol)
...

$ uname -a 
Linux ... 3.0.0-8-generic #10-Ubuntu SMP Fri Aug 5 23:54:15 UTC 2011 x86_64 
x86_64 x86_64 GNU/Linux

The small patch attached sets the family to AF_INET.

For some users, interpreting httperf's output can be difficult.

Since a large part of doing so is assuring that the test ran successfully
(i.e., there were no client-side errors), it strikes me that it would
simplify things considerably if we added a command-line flag that, when
present, would instruct httperf to abort with an appropriate complaint if;

a) total CPU is less than, say, 90%, or
b) any errors *other than* client-timeout are seen

This would make it significantly easier for operators to interpret the output.

Optionally, if the flag is present and the test run is successful, we could
suppress the CPU and error information (perhaps just leaving the total), to
make output cleaner.

httperf --timeout=100 --client=0/1 --server=192.168.54.100 --port=80 --uri=/ --rate=10 --send-buffer=4096 --recv-buffer=16384 --ssl-protocol=auto --num-conns=100 --num-calls=1

Maximum connect burst length: 1

Total: connections 100 requests 100 replies 0 test-duration 109.923 s

Connection rate: 0.9 conn/s (1099.2 ms/conn, <=100 concurrent connections)
Connection time [ms]: min 0.0 avg 0.0 max 0.0 median 0.0 stddev 0.0
Connection time [ms]: connect 22.7
Connection length [replies/conn]: 0.000

Request rate: 0.9 req/s (1099.2 ms/req)
Request size [B]: 67.0

Reply rate [replies/s]: min 0.0 avg 0.0 max 0.0 stddev 0.0 (21 samples)
Reply time [ms]: response 0.0 transfer 0.0
Reply size [B]: header 0.0 content 0.0 footer 0.0 (total 0.0)
Reply status: 1xx=0 2xx=0 3xx=0 4xx=0 5xx=0

CPU time [s]: user 46.50 system 63.26 (user 42.3% system 57.5% total 99.8%)
Net I/O: 0.1 KB/s (0.0*10^6 bps)

Errors: total 100 client-timo 100 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

What steps will reproduce the problem?
1. MANWIDTH=70 man httperf | awk '{ if(FNR == 13 || FNR == 46){ print $NL } }'

What is the expected output? What do you see instead?

Expected:
       calls N] [--num-conns N] [--period [d|u|e]T1[,T2]] [--port N]
       httperf --hog --server www --num-conns 100 --ra 10 --timeout 5


Actual:
       calls N] [--num-conns N] [--period [d|u|e]T1[,T2]] [--port N]
       httperf --hog --server www --num-conn 100 --ra 10 --timeout 5


What version of the product are you using? On what operating system?

0.9.0 on Archlinux

Please provide any additional information below.

This might be a way to fix it:
sed -i 's/conn /conns /'

What steps will reproduce the problem?
1. Run httperf as follows httperf  --timeout=10   --server=*host*  --port=80 
--wlog=y,requests_httperf_Hindi.txt --uri=/ --rate=200 --num-conns=2000 
--num-calls=1 --client=1/3 
2. outputs Segmentation fault: 11
3.

What is the expected output? What do you see instead?
I expect the test to go on but instead it segfaults.
However if I lower rate to some 100 it does not segfault.



What version of the product are you using? On what operating system?
v0.9

Please provide any additional information below.
ulimit on the host
ulimit -n
10000
uname -a
Darwin degreethem-lm 11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 
PDT 2012; root:xnu-1699.32.7~1/RELEASE_X86_64 x86_64

What steps will reproduce the problem?
1. use httperf from a cloud of amazon EC2 instances on Debian, with autobench
2.
3.

What is the expected output? What do you see instead?

each time I run my autobench tests, at least one instance of httperf gets stuck 
into an endless loop. debug enabled, here is the result :
every 5 seconds it fires these 2 lines

event_signal: EV_HOSTNAME_LOOKUP_START (obj=(nil),arg=2b80e9e8)
timer_schedule: t=0x81ca5f8, delay=5s, subject=0

thanks for your help,
sam

aclocal: warning: couldn't open directory 'm4': No such file or directory
configure.ac:21: error: possibly undefined macro: AC_PROG_LIBTOOL
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1

I installed httperf on my MBP running Mac OSX 10.6.4
Installed httperf and autobench. When I executed the command:

autobench --single_host --host1 google.com --uri1 / --low_rate 5 --high_rate 20 
--rate_step 10 --num_call 10

Below is the error result:

Zero replies received, test invalid: rate 15
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.
Use of uninitialized value in concatenation (.) or string at 
/usr/local/bin/autobench line 308.

If you specify --print-reply and the response body is large, the headers
will print repeatedly as the content streams out.

I think this is because print_reply() is called every time send_raw_data fires.

E.g.,

SH0:PUT /test.jpg HTTP/1.1
SH0:Host: www.example.net
SH0:Content-Type: image/jpeg
SH0:Content-Length: 55672
SH0:User-Agent: httperf/0.9.0
SH0:
SS0: header 148 content 55672
SH0:PUT /test.jpg HTTP/1.1
SH0:Host: www.example.net
SH0:Content-Type: image/jpeg
SH0:Content-Length: 55672
SH0:User-Agent: httperf/0.9.0
SH0:
SS0: header 148 content 51476
SH0:PUT /test.jpg HTTP/1.1
SH0:Host: www.example.net
SH0:Content-Type: image/jpeg
SH0:Content-Length: 55672
SH0:User-Agent: httperf/0.9.0
SH0:

httperf does not recognize "Connection: close" headers, it assumes Web 
servers that claim to be "HTTP/1.1" to support persistent connections.
Likewise, it assumes servers that claim to be "HTTP/1.0" do not support
persistent connections.

From RFC2616 section 14.10;

HTTP/1.1 defines the "close" connection option for the sender to signal
that the connection will be closed after completion of the response. For
example, Connection: close in either the request or the response header
fields indicates that the connection SHOULD NOT be considered `persistent'
(section 8.1) after the current request/response is complete. HTTP/1.1
applications that do not support persistent connections MUST include the
"close" connection option in every message.

What steps will reproduce the problem?
1. Run `httperf --server=google.com --uri=/ --num-calls=3 --burst-length=2 -v`

What is the expected output? What do you see instead?
Very quick exit. Actual result is that httperf hangs.

What version of the product are you using? On what operating system?
Tested on 0.9.0

Currently httperf v0.9.0 does not support SSL client certificate
authentication, server certificate verification, or choosing the SSL
protocol version.  This patch adds the following:

1) Client SSL certificate authentication
   - add --ssl-certificate option (required argument file name) to specify
the certificate
   - add --ssl-key option (required argument file name) to specify the key
   - loads certificate and key into the SSL context and performs a
consistency check
2) Server certificate verification
   - add --ssl-verify option (optional arguments "no" and "yes") to control
verification (off by default)
   - add --ssl-ca-file and --ssl-ca-path options (required arguments file
and path names) to set custom certificate authority file and path
   - load system default certificate path (e.g., /etc/ssl/certs)
3) Choose SSL protocol version
   - add --ssl-protocol option (required argument of "auto", "SSLv2",
"SSLv3", or "TLSv1") to choose protocol; "auto" (default) chooses
SSLv23_client_method

The patch modifies httperf.h and httperf.c, adding the appropriate option
processing and OpenSSL code, as well as reporting these options in help
text (--help) and the command line summary when running the program.  It
has been tested on MacOS X (10.4), Ubuntu 9.04, and Solaris 10 x86 using
gcc3.  It is known to operate properly against Apache 2.2/mod_ssl, pound,
and Ruby WEBrick servers.

Note: this changes the default SSL protocol from SSLv3_client_method to
SSLv23_client_method.  Other new options (e.g. --ssl-verify) are off by
default.

For my testing, I need to set my own Host header, so I disable the
auto-generated Host header with --no-host-hdr and add in my own Host header
among other additional headers I need with the --add-header option.

With that, the auto-generated Host header is gone, but left with an empty
line "\r\n" (see the output with the empty line below), which with another
"\r\n" signals end of header section in HTTP. As a result, my own
additional headers are treated as the body of the HTTP request and thus
ignored.

SH0:GET /folder/hello.txt HTTP/1.1
SH0:User-Agent: httperf/0.9.0
SH0:
SH0:My-Header: some text
SH0:Host: www.mydomain.com
SH0:

    $ mkdir build
    $ cd build
    $ SRCDIR/configure
    $ make
    $ make install

I tried the following test, but I was getting a lot of fd-unavail errors. 

httperf --server myIP --port 80 --num-conns 90000 --rate 1000 --method GET 
--uri myURL


To fix that, I increased the number of FD_SETSIZE on 
/etc/include/bits/typesizes.h to 65535.


Re-running the same test, I get a buffer overflow detected error. 

*** buffer overflow detected ***: httperf terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7f9228bf3807]
/lib/x86_64-linux-gnu/libc.so.6(+0x109700)[0x7f9228bf2700]
/lib/x86_64-linux-gnu/libc.so.6(+0x10a7be)[0x7f9228bf37be]
httperf[0x404054]
httperf[0x404e9f]
httperf[0x406943]
httperf[0x406bc1]
httperf[0x40638f]
httperf[0x405057]
httperf[0x40285e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7f9228b0a76d]
httperf[0x4038f1]
======= Memory map: ========
00400000-00410000 r-xp 00000000 08:12 6424320                            
/usr/lo      cal/bin/httperf
0060f000-00610000 r--p 0000f000 08:12 6424320                            
/usr/lo      cal/bin/httperf
00610000-00611000 rw-p 00010000 08:12 6424320                            
/usr/lo      cal/bin/httperf
00611000-006a4000 rw-p 00000000 00:00 0
024bd000-02628000 rw-p 00000000 00:00 0                                  [heap]
7f92284b8000-7f92284cd000 r-xp 00000000 08:12 1969821                    
/lib/x8      6_64-linux-gnu/libgcc_s.so.1
7f92284cd000-7f92286cc000 ---p 00015000 08:12 1969821                    
/lib/x8      6_64-linux-gnu/libgcc_s.so.1
7f92286cc000-7f92286cd000 r--p 00014000 08:12 1969821                    
/lib/x8      6_64-linux-gnu/libgcc_s.so.1
7f92286cd000-7f92286ce000 rw-p 00015000 08:12 1969821                    
/lib/x8      6_64-linux-gnu/libgcc_s.so.1
7f92286ce000-7f92286e4000 r-xp 00000000 08:12 1969911                    
/lib/x8      6_64-linux-gnu/libz.so.1.2.3.4
7f92286e4000-7f92288e3000 ---p 00016000 08:12 1969911                    
/lib/x8      6_64-linux-gnu/libz.so.1.2.3.4
7f92288e3000-7f92288e4000 r--p 00015000 08:12 1969911                    
/lib/x8      6_64-linux-gnu/libz.so.1.2.3.4
7f92288e4000-7f92288e5000 rw-p 00016000 08:12 1969911                    
/lib/x8      6_64-linux-gnu/libz.so.1.2.3.4
7f92288e5000-7f92288e7000 r-xp 00000000 08:12 1969813                    
/lib/x8      6_64-linux-gnu/libdl-2.15.so
7f92288e7000-7f9228ae7000 ---p 00002000 08:12 1969813                    
/lib/x8      6_64-linux-gnu/libdl-2.15.so
7f9228ae7000-7f9228ae8000 r--p 00002000 08:12 1969813                    
/lib/x8      6_64-linux-gnu/libdl-2.15.so
7f9228ae8000-7f9228ae9000 rw-p 00003000 08:12 1969813                    
/lib/x8      6_64-linux-gnu/libdl-2.15.so
7f9228ae9000-7f9228c9e000 r-xp 00000000 08:12 1969800                    
/lib/x8      6_64-linux-gnu/libc-2.15.so
7f9228c9e000-7f9228e9d000 ---p 001b5000 08:12 1969800                    
/lib/x8      6_64-linux-gnu/libc-2.15.so
7f9228e9d000-7f9228ea1000 r--p 001b4000 08:12 1969800                    
/lib/x8      6_64-linux-gnu/libc-2.15.so
7f9228ea1000-7f9228ea3000 rw-p 001b8000 08:12 1969800                    
/lib/x8      6_64-linux-gnu/libc-2.15.so
7f9228ea3000-7f9228ea8000 rw-p 00000000 00:00 0
7f9228ea8000-7f9228fa3000 r-xp 00000000 08:12 1969832                    
/lib/x8      6_64-linux-gnu/libm-2.15.so
7f9228fa3000-7f92291a2000 ---p 000fb000 08:12 1969832                    
/lib/x8      6_64-linux-gnu/libm-2.15.so
7f92291a2000-7f92291a3000 r--p 000fa000 08:12 1969832                    
/lib/x8      6_64-linux-gnu/libm-2.15.so
7f92291a3000-7f92291a4000 rw-p 000fb000 08:12 1969832                    
/lib/x8      6_64-linux-gnu/libm-2.15.so
7f92291a4000-7f9229355000 r-xp 00000000 08:12 1966085                    
/lib/x8      6_64-linux-gnu/libcrypto.so.1.0.0
7f9229355000-7f9229555000 ---p 001b1000 08:12 1966085                    
/lib/x8      6_64-linux-gnu/libcrypto.so.1.0.0
7f9229555000-7f9229570000 r--p 001b1000 08:12 1966085                    
/lib/x8      6_64-linux-gnu/libcrypto.so.1.0.0
7f9229570000-7f922957b000 rw-p 001cc000 08:12 1966085                    
/lib/x8      6_64-linux-gnu/libcrypto.so.1.0.0
7f922957b000-7f922957f000 rw-p 00000000 00:00 0
7f922957f000-7f92295d3000 r-xp 00000000 08:12 1966092                    
/lib/x8      6_64-linux-gnu/libssl.so.1.0.0
7f92295d3000-7f92297d3000 ---p 00054000 08:12 1966092                    
/lib/x8      6_64-linux-gnu/libssl.so.1.0.0
7f92297d3000-7f92297d6000 r--p 00054000 08:12 1966092                    
/lib/x8      6_64-linux-gnu/libssl.so.1.0.0
7f92297d6000-7f92297dc000 rw-p 00057000 08:12 1966092                    
/lib/x8      6_64-linux-gnu/libssl.so.1.0.0
7f92297dc000-7f92297dd000 rw-p 00000000 00:00 0
7f92297dd000-7f92297ff000 r-xp 00000000 08:12 1969780                    
/lib/x8      6_64-linux-gnu/ld-2.15.so
7f92299e7000-7f92299ec000 rw-p 00000000 00:00 0
7f92299fb000-7f92299ff000 rw-p 00000000 00:00 0
7f92299ff000-7f9229a00000 r--p 00022000 08:12 1969780                    
/lib/x8      6_64-linux-gnu/ld-2.15.so
7f9229a00000-7f9229a02000 rw-p 00023000 08:12 1969780                    
/lib/x8      6_64-linux-gnu/ld-2.15.so
7fff70a79000-7fff70a9a000 rw-p 00000000 00:00 0                          [stack]
7fff70bff000-7fff70c00000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  
[vsysca      ll]

I'm using httperf 0.9.0 downloaded from this site on Ubuntu 12.04 LTS 64 bits.


If I set the FD_SETSIZE to 1024, the error disappear (but I get again lots of 
fd-unavails).

To set the FD_SETSIZE I followed these guides, but without any improvement.

http://aunixlover.blogspot.com/2012/02/how-to-configure-clients-and-servers.html

http://www.lkj.net/2011/07/increasing-httperf-maximum-number-of-open-descriptors
-from-1024-to-65535/#comment-3888

The wsesslog file has no option to add http headers to individual requests. 
Attached is a patch that will allow up to 16k of arbitrary headers to be added.

Example usage: 
# session 1 definition (this is a comment)
/foo.html think=2.0 headers='Cookie: CO=blahblahblah;'
      /pict1.gif
      /pict2.gif
/foo2.html method=POST contents='Post data' headers='Cookie: CO=foo;'
      /pict3.gif
      /pict4.gif

Based on the patch from 
http://www.overset.com/2008/03/27/load-test-ajax-applications-with-httperf/ but 
with modest improvements.

It is useful to set arbitrary source address when benchmarking some
configurations. I've made a quick patch (no docs, source code only, sorry). 

      while ((ssl_err = SSL_connect(s->ssl)) == -1) {
                int reason = SSL_get_error(s->ssl, ssl_err);
                if (reason != SSL_ERROR_WANT_READ &&
                    reason != SSL_ERROR_WANT_WRITE) {
                        fprintf(stderr,
                            "%s: failed to connect to SSL server (err=%d, reason=%d)\n",
                            prog_name, ssl_err, reason);
                        ERR_print_errors_fp(stderr);
                        exit(-1);
                }
        }

What steps will reproduce the problem?
1. Create a wlog uri file with a forward slash in it
2. Issue a single request against yahoo.com (as an example only):
httperf --server=www.yahoo.com --wlog=n,uri.txt --rate=1 --num-conns=1 
--num-calls=1  --print-request --print-reply=header

What is the expected output? What do you see instead?
I would expect to see results, but instead I see the above error

What version of the product are you using? On what operating system?
0.9.0, ubuntu 10

Please provide any additional information below.
When issuing the above command, the request headers have the "HTTP/1.1" on a 
second line as opposed to the first line.  The web server returns no status 
lines when this happens.

If you issue the same test with the --uri parameter instead of --wlog:
httperf --server=www.yahoo.com --uri=/ --num-conns=1 --num-calls=1 rate=1 
--print-request --print-reply=header

then the test succeeds as expected.  Here is the comparison between the request 
headers between the two requests:

SH0:GET /
SH0: HTTP/1.1
SH0:User-Agent: httperf/0.9.0
SH0:Host: www.yahoo.com
SH0:
SS0: header 67 content 0


SH0:GET / HTTP/1.1
SH0:User-Agent: httperf/0.9.0
SH0:Host: www.yahoo.com
SH0:
SS0: header 66 content 0

and the difference in the reply headers, the first causing the error:

RH0:<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
RH0:   "http://www.w3.org/TR/html4/strict.dtd">
RH0:<html lang="en-US" class="y-fp-bg y-fp-pg-grad  bkt701">


RH0:HTTP/1.1 200 OK
RH0:Date: Thu, 13 Jan 2011 20:12:28 GMT
...

Fixed typo "You can choose to loop on ***the*** list of URIs by using the 
following"

-- What steps will reproduce the problem?
run the following command:
httperf --hog --rate=500 --num-conns=200000 --timeout=5 --uri=/img/default.jpeg 
--num-calls=10 --server=svr-web

-- What is the expected output? What do you see instead?

The test shall be finished within about 400 seconds (200000 / 500), but it get 
stuck.   After a while, I found that there is no network traffic from client to 
server, and netstat shows that there are more than 1000 sockets in CLOSE_WAIT 
state.   Note that if I use --num-conns with a smaller number, e.g. 20000, 
there is no problem at all.  I guess this is a problem in httperf on handling 
socket close operation?

-- What version of the product are you using? On what operating system?

Ubuntu 10.04 (both server and client) on Q8400 with 4G memory.   Note that the 
server is also running on a desktop (gnome) version of ubuntu, not server 
edition.  Also I have tuned file descriptor limit on the client and compiled 
httperf on it.

root@host:web# httperf --hog --num-calls 10000 --port 3000 --server localhost --wsesslog=100000,0,file
httperf --hog --client=0/1 --server=localhost --port=3000 --uri=/ --send-buffer=4096 --recv-buffer=16384 --wsesslog=100000,0.000,file

My name is Christopher Torres, I'm an undergraduate student at the
University of Puerto at Bayamon. Currently I'm involved on an undergraduate
research initiative lead by Dr. Juan Sola Sloan
(http://www.uprb.edu/profesor/jsola/) working on httperf. Our current goal
is to provide functionality that allows dumping test results into a file
for further performance analysis. Another fellow student is working on a
tool that uses this file as input and generates some cool graphs in a
webpage. How can we contribute to the whole httperf project with our findings?

Thank you for your time,
Christopher Torres

xerrot at gmail dot com

if (setrlimit (RLIMIT_NOFILE, &rlimit) < 0)

I don't think it makes sense to assume that the server is "localhost" and
the URI is "/" if they're not supplied on the command line; perf testing
the same box as you run the clients on is bad practice.

Instead, these arguments (or a suitable logfile generator) should be
required, and httperf should fail if it can't find an explicit
specification of the test URI.

httperf will always emit the httperf user-agent, even when set explicitly
in arguments. This makes it difficult to test when UA is important to the
server.

Two possible fixes;

1) make it possible to explicitly override the UA on the command line. It
would probably also be necessary to allow logs (e.g., wlog) to override it.

2) omit the UA, unless it's specified.

#2 is probably easier, but may cause problems if people don't remember to
set a UA.