moviepilot / stresser

A ruby wrapper around httperf. It parses httperf's output and exports the output of multiple runs as a csv file, which can then be transformed into many useful images of graphs.

Stresser

This gem is a wrapper around the httperf command which can put all types of loads on a webserver. It's like apachebench, but you can replay log files, define sessions, and so forth.

This gem calls httperf many times with different concurrency settings and parses httperf's output into a csv file, that you can then use to visualize your application's performance at different concurrency levels

Sample graphs

Here's a collection of graphs that this gem currently creates (though you can create your own by creating a YML file that maps columns from the generated csv file to labels for the image).

Installation

First install the gem

$ gem install stresser

Configuration

Please refer to the supplied sample.conf on how to configure stresser. Also, see man httperf as all options in sample.conf beginning with httperf_ go directly to the httperf commands.

Examples

Stresstest

You can call stresser from the command line:

$ stresser -c your_app.conf -o /tmp/stress/result.csv
... lots of httperf output...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
Great, now create a graph with
  stresser-grapher -o /tmp/stress /tmp/stress/result.csv
$

You will see the output of the httperf commands that are issued, and a full report will be written to result.csv.

Creating graphs

When you're done, you can create a graph of your testrun like this:

$ stresser-grapher -o /tmp/stress /tmp/stress/result.csv 
Generating stati_per_second to /tmp/stress/2010_10_25_17_28_stati_per_second.png...
Generating replies_per_second to /tmp/stress/2010_10_25_17_28_replies_per_second.png...
Generating errors to /tmp/stress/2010_10_25_17_28_errors.png...
Generating connection_time to /tmp/stress/2010_10_25_17_28_connection_time.png...
Generating cpu to /tmp/stress/2010_10_25_17_28_cpu.png...
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
Great, now open the images with
  open /tmp/2010_10_25_17_28*.png
$

Log generator

As a little helper to generate log files defining some session workload that requires different urls, stresser-loggen is supplied. Just create a log template named mylog.tpl like this

# My session workload
/users/{{n}}
  /images/foo.gif
  /images/bar.gif
/users{{n}}/dashboard

And then use stresser-loggen to reproduce these lines as often as you like:

stresser-loggen mylog.tpl 100 > mylog.conf

The {{n}} will be replaced with the numbers 0-99.

Tests

Run rake spec to run them tests. Currently, only httperf's output of a session based replay log is parsed, but I will add more.

Thanks

Stresser is based on igvita's autoperf driver for httperf.

stresser's Issues

Stresser doesn't work when option httperf_add-header is used

I tried to use add-header like this: httperf_add-header = "Referer: http://site.com\n" and stresser doesn't work.
If I comment out this option stresser works just fine.
The version I'm using was installed with gem install ...

FasterCSV plus support for Ruby 1.9's m17n encoding engine. (NotImplementedError)

I also get this on ruby 1.9.3 bundled with Ubuntu 12.04

$ stresser-grapher -o /tmp /tmp/res.csv                                                                                  1 ↵  
Generating stati_per_second to /tmp/2012_05_27_22_13_stati_per_second.png...
/var/lib/gems/1.9.1/gems/fastercsv-1.5.5/lib/faster_csv.rb:2001: warning: already initialized constant FCSV
/var/lib/gems/1.9.1/gems/fastercsv-1.5.5/lib/faster_csv.rb:13:in `const_missing': Please switch to Ruby 1.9's standard CSV library.  It's FasterCSV plus support for Ruby 1.9's m17n encoding engine. (NotImplementedError)

options without values not handled

I have to add options like --ssl and --ssl-no-reuse, I tried to write into the config file the following:
httperf_ssl (doesn't see it)
httperf_ssl = (httperf: option '--ssl' doesn't allow an argument)
httperf_ssl = nil (httperf: option '--ssl' doesn't allow an argument)
httperf_ssl = "nil" (httperf: option '--ssl' doesn't allow an argument)

I don't know if I'm doing something wrong...
I've never used ruby before but the httperf.rb line
'''
httperf_opt = conf.keys.grep(/httperf/).collect {|k| conf[k] == "nil" ? "--#{k.gsub(/httperf_/, '')}" : "--#{k.gsub(/httperf_/, '')}=#{conf[k]}"}.join(" ")
'''
looks fine to me, maybe it's something into the mp_perf.rb parsing function

`CSV': undefined method `instance' for Ruport::Formatter::CSV:Class

I don't know if it is a stresser bug, but the csv isn't generated

➜  ~  stresser -c Code/webbenchmark/railstest.local -o /tmp/res.csv

Net I/O: 77.1 KB/s (0.6*10^6 bps)

Errors: total 11 client-timo 11 socket-timo 0 connrefused 0 connreset 0
Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0

/Users/grigio/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/csv.rb:2324:in CSV': undefined methodinstance' for Ruport::Formatter::CSV:Class (NoMethodError)
from /Users/grigio/.rvm/gems/ruby-1.9.2-p290/gems/ruport-1.6.3/lib/ruport/formatter/csv.rb:59:in csv_writer' from /Users/grigio/.rvm/gems/ruby-1.9.2-p290/gems/ruport-1.6.3/lib/ruport/formatter/csv.rb:70:inbuild_table_header'
from /Users/grigio/.rvm/gems/ruby-1.9.2-p290/gems/ruport-1.6.3/lib/ruport/controller.rb:598:in maybe' from /Users/grigio/.rvm/gems/ruby-1.9.2-p290/gems/ruport-1.6.3/lib/ruport/controller.rb:584:inblock in execute_stages'
from /Users/grigio/.rvm/gems/ruby-1.9.2-p290/gems/ruport-1.6.3/lib/ruport/controller.rb:583:in `each'

README command options -c

The README is missing the -c, it should be:

stresser -c your_app.conf -o /tmp/stress/result.csv

Without -c I get Error: argument --output-file must be a writeable file.

I'm happy to submit a small pull request if you think I am right.

MacOS, JRuby, Stresser 0.5