zombocom / derailed_benchmarks

Go faster, off the Rails - Benchmarks for your whole Rails app

Derailed Benchmarks

A series of things you can use to benchmark a Rails or Ruby app.

Compatibility/Requirements

For some benchmarks, not all, you'll need to verify you have a working version of curl on your OS:

$ which curl
/usr/bin/curl
$ curl -V
curl 7.64.1 #...

Install

Put this in your gemfile:

gem 'derailed_benchmarks', group: :development

Then run $ bundle install.

While executing your commands you may need to use bundle exec before typing the command.

To use all profiling methods available also add:

gem 'stackprof', group: :development

You must be using Ruby 2.1+ to install these libraries. If you're on an older version of Ruby, what are you waiting for?

Use

There are two ways to benchmark an app. Derailed can either try to boot your web app and run requests against it while benchmarking, or it can statically give you more information about the dependencies that are in your Gemfile. Booting your app will always be more accurate, but if you cannot get your app to run in production locally, you'll still find the static information useful.

Static Benchmarking

This section covers how to get memory information from your Gemfile without having to boot your app.

All commands in this section will begin with $ derailed bundle:

For more information on the relationship between memory and performance please read/watch How Ruby Uses Memory.

Memory used at Require time

Each gem you add to your project can increase your memory at boot. You can get visibility into the total memory used by each gem in your Gemfile by running:

$ bundle exec derailed bundle:mem

This will load each of your gems in your Gemfile and see how much memory they consume when they are required. For example if you're using the mail gem. The output might look like this

$ bundle exec derailed bundle:mem
TOP: 54.1836 MiB
  mail: 18.9688 MiB
    mime/types: 17.4453 MiB
    mail/field: 0.4023 MiB
    mail/message: 0.3906 MiB
  action_view/view_paths: 0.4453 MiB
    action_view/base: 0.4336 MiB

Aside: A "MiB", which is the IEEE and IEC symbol for Mebibyte, is 2²⁰ bytes / 1024 Kibibytes (which are in turn 1024 bytes).

Here we can see that mail uses 18MiB, with the majority coming from mime/types. You can use this information to prune out large dependencies you don't need. Also if you see a large memory use by a gem that you do need, please open up an issue with that library to let them know (be sure to include reproduction instructions). Hopefully as a community we can identify memory hotspots and reduce their impact. Before we can fix performance problems, we need to know where those problems exist.

By default this task will only return results from the :default and "production" groups. If you want a different group you can run with.

$ bundle exec derailed bundle:mem development

You can use CUT_OFF=0.3 to only show files that have above a certain memory usage, this can be used to help eliminate noise.

Note: This method won't include files in your own app, only items in your Gemfile. For that you'll need to use bundle exec derailed exec mem. See below for more info.

The same file may be required by several libraries, since Ruby only requires files once, the cost is only associated with the first library to require a file. To make this more visible duplicate entries will list all the parents they belong to. For example both mail and fog require `mime/types. So it may show up something like this in your app:

$ bundle exec derailed bundle:mem
TOP: 54.1836 MiB
  mail: 18.9688 MiB
    mime/types: 17.4453 MiB (Also required by: fog/storage)
    mail/field: 0.4023 MiB
    mail/message: 0.3906 MiB

That way you'll know that simply removing the top level library (mail) would not result in a memory reduction. The output is truncated after the first two entries:

fog/core: 0.9844 MiB (Also required by: fog/xml, fog/json, and 48 others)
fog/rackspace: 0.957 MiB
fog/joyent: 0.7227 MiB
  fog/joyent/compute: 0.7227 MiB

If you want to see everything that requires fog/core you can run CUT_OFF=0 bundle exec derailed bundle:mem to get the full output that you can then grep through manually.

Update: While mime/types looks horrible in these examples, it's been fixed. You can add this to the top of your gemfile for free memory:

gem 'mime-types', [ '~> 2.6', '>= 2.6.1' ], require: 'mime/types/columnar'

Objects created at Require time

To get more info about the objects, using memory_profiler, created when your dependencies are required you can run:

$ bundle exec derailed bundle:objects

This will output detailed information about objects created while your dependencies are loaded

Measuring objects created by gems in groups [:default, "production"]
Total allocated 433895
Total retained 100556

allocated memory by gem
-----------------------------------
  24369241  activesupport-4.2.1
  15560550  mime-types-2.4.3
   8103432  json-1.8.2

Once you identify a gem that creates a large amount of memory using $ bundle exec derailed bundle:mem you can pull that gem into it's own Gemfile and run $ bundle exec derailed bundle:objects to get detailed information about it. This information can be used by contributors and library authors to identify and eliminate object creation hotspots.

By default this task will only return results from the :default and "production" groups. If you want a different group you can run with.

$ bundle exec derailed bundle:objects development

Note: This method won't include files in your own app, only items in your Gemfile. For that you'll need to use bundle exec derailed exec objects. See below for more info.

Dynamic app Benchmarking

This benchmarking will attempt to boot your Rails app and run benchmarks against it. Unlike the static benchmarking with $ bundle exec derailed bundle:* these will include information about your specific app. The pro is you'll get more information and potentially identify problems in your app code, the con is that it requires you to be able to boot and run your application in a production environment locally, which for some apps is non-trivial.

You may want to check out mini-profiler, here's a mini-profiler walkthrough. It's great and does slightly different benchmarking than what you'll find here.

Running in Production Locally.

Before you want to attempt any dynamic benchmarks, you'll need to boot your app in production mode. We benchmark using production because it is close to your deployed performance. This section is more a collection of tips rather than a de-facto tutorial.

For starters try booting into a console:

$ RAILS_ENV=production rails console

You may get errors, complaining about not being able to connect to the production database. For this, you can either create a local database with the name of your production database, or you can copy the info from your development group to your production group in your database.yml.

You may be missing environment variables expected in production such as SECRET_KEY_BASE. For those you can either commit them to your .env file (if you're using one). Or add them directly to the command:

$ SECRET_KEY_BASE=foo RAILS_ENV=production rails console

Once you can boot a console in production, you'll need to be able to boot a server in production

$ RAILS_ENV=production rails server

You may need to disable enforcing SSL or other domain restrictions temporarily. If you do these, don't forget to add them back in before deploying any code (eek!).

You can get information from STDOUT if you're using rails_12factor gem, or from log/production.log by running

$ tail -f log/production.log

Once you've fixed all errors and you can run a server in production, you're almost there.

Running Derailed Exec

You can run commands against your app by running $ derailed exec. There are sections on setting up Rack and using authenticated requests below. You can see what commands are available by running:

$ bundle exec derailed exec --help
  $ derailed exec perf:allocated_objects  # outputs allocated object diff after app is called TEST_COUNT times
  $ derailed exec perf:app  # runs the performance test against two most recent commits of the current app
  $ derailed exec perf:gc  # outputs GC::Profiler.report data while app is called TEST_COUNT times
  $ derailed exec perf:heap  # heap analyzer
  $ derailed exec perf:ips  # iterations per second
  $ derailed exec perf:library  # runs the same test against two different branches for statistical comparison
  $ derailed exec perf:mem  # show memory usage caused by invoking require per gem
  $ derailed exec perf:mem_over_time  # outputs memory usage over time
  $ derailed exec perf:objects  # profiles ruby allocation
  $ derailed exec perf:stackprof  # stackprof
  $ derailed exec perf:test  # hits the url TEST_COUNT times
  $ derailed exec perf:heap_diff  # three heaps generation for comparison

Instead of going over each command we'll look at common problems and which commands are best used to diagnose them. Later on we'll cover all of the environment variables you can use to configure derailed benchmarks in it's own section.

Is my app leaking memory?

If your app appears to be leaking ever increasing amounts of memory, you'll want to first verify if it's an actual unbound "leak" or if it's just using more memory than you want. A true memory leak will increase memory use forever, most apps will increase memory use until they hit a "plateau". To diagnose this you can run:

$ bundle exec derailed exec perf:mem_over_time

This will boot your app and hit it with requests and output the memory to stdout (and a file under ./tmp). It may look like this:

$ bundle exec derailed exec perf:mem_over_time
Booting: production
Endpoint: "/"
PID: 78675
103.55078125
178.45703125
179.140625
180.3671875
182.1875
182.55859375
# ...
183.65234375
183.26171875
183.62109375

Here we can see that while the memory use is increasing, it levels off around 183 MiB. You'll want to run this task using ever increasing values of TEST_COUNT= for example

$ TEST_COUNT=5000 bundle exec derailed exec perf:mem_over_time
$ TEST_COUNT=10_000 bundle exec derailed exec perf:mem_over_time
$ TEST_COUNT=20_000 bundle exec derailed exec perf:mem_over_time

Adjust your counts appropriately so you can get results in a reasonable amount of time. If your memory never levels off, congrats! You've got a memory leak! I recommend copying and pasting values from the file generated into google docs and graphing it so you can get a better sense of the slope of your line.

If you don't want it to generate a tmp file with results run with SKIP_FILE_WRITE=1.

If you're pretty sure that there's a memory leak, but you can't confirm it using this method. Look at the environment variable options below, you can try hitting a different endpoint etc.

Dissecting a Memory Leak

If you've identified a memory leak, or you simply want to see where your memory use is coming from you'll want to use

$ bundle exec derailed exec perf:objects

This task hits your app and uses memory_profiler to see where objects are created. You'll likely want to run once, then run it with a higher TEST_COUNT so that you can see hotspots where objects are created on EVERY request versus just maybe on the first.

$ TEST_COUNT=10 bundle exec derailed exec perf:objects

This is an expensive operation, so you likely want to keep the count lowish. Once you've identified a hotspot read how ruby uses memory for some tips on reducing object allocations.

This is similar to $ bundle exec derailed bundle:objects however it includes objects created at runtime. It's much more useful for actual production performance debugging, the other is more useful for library authors to debug.

I want a Heap Dump

If you're still struggling with runtime memory you can generate a heap dump that can later be analyzed using heapy.

$ bundle exec derailed exec perf:heap
Booting: production
Heap file generated: "tmp/2015-10-01T12:31:03-05:00-heap.dump"

Analyzing Heap
==============
Generation:  0 object count: 209307
Generation: 35 object count: 31236
Generation: 36 object count: 36705
Generation: 37 object count: 1301
Generation: 38 object count: 8

Try uploading "tmp/2015-10-01T12:31:03-05:00-heap.dump" to http://tenderlove.github.io/heap-analyzer/

For more help on getting data from a heap dump see

$ heapy --help

I want more heap dumps

When searching for a leak, you can use heap dumps for comparison to see what is retained. See Analyzing memory heaps (inspired from SamSaffron's original idea) for a clear example. You can generate 3 dumps (one every TEST_COUNT calls) using the next command:

$ bundle exec derailed exec perf:heap_diff
Endpoint: "/"
Running 1000 times
Heap file generated: "tmp/2021-05-06T15:19:26+02:00-heap-0.ndjson"
Running 1000 times
Heap file generated: "tmp/2021-05-06T15:19:26+02:00-heap-1.ndjson"
Running 1000 times
Heap file generated: "tmp/2021-05-06T15:19:26+02:00-heap-2.ndjson"

Diff
====
Retained STRING 90 objects of size 4790/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/rack-2.2.3/lib/rack/utils.rb:461
Retained ICLASS 20 objects of size 800/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/sinatra-contrib-2.0.8.1/lib/sinatra/namespace.rb:198
Retained DATA 20 objects of size 1360/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/2.7.0/monitor.rb:238
Retained STRING 20 objects of size 800/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/rack-protection-2.0.8.1/lib/rack/protection/xss_header.rb:20
Retained STRING 10 objects of size 880/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/newrelic_rpm-5.4.0.347/lib/new_relic/agent/transaction.rb:890
Retained CLASS 10 objects of size 4640/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/sinatra-contrib-2.0.8.1/lib/sinatra/namespace.rb:198
Retained IMEMO 10 objects of size 480/91280 (in bytes) at: /Users/ulysse/.rbenv/versions/2.7.2/lib/ruby/gems/2.7.0/gems/sinatra-2.0.8.1/lib/sinatra/base.rb:1017
...

Run `$ heapy --help` for more options

Also read https://medium.com/klaxit-techblog/tracking-a-ruby-memory-leak-in-2021-9eb56575f731#875b to understand better what you are reading.

Memory Is large at boot.

Ruby memory typically goes in one direction, up. If your memory is large when you boot the application it will likely only increase. In addition to debugging memory retained from dependencies obtained while running $ derailed bundle:mem you'll likely want to see how your own files contribute to memory use.

This task does essentially the same thing, however it hits your app with one request to ensure that any last minute require-s have been called. To execute you can run:

$ bundle exec derailed exec perf:mem

TOP: 54.1836 MiB
  mail: 18.9688 MiB
    mime/types: 17.4453 MiB
    mail/field: 0.4023 MiB
    mail/message: 0.3906 MiB
  action_view/view_paths: 0.4453 MiB
    action_view/base: 0.4336 MiB

You can use CUT_OFF=0.3 to only show files that have above a certain memory usage, this can be used to help eliminate noise.

If your application code is extremely large at boot consider using $ derailed exec perf:objects to debug low level object creation.

My app is Slow

Well...aren't they all. If you've already looked into decreasing object allocations, you'll want to look at where your app is spending the most amount of code. Once you know that, you'll know where to spend your time optimising.

One technique is to use a "sampling" stack profiler. This type of profiling looks at what method is being executed at a given interval and records it. At the end of execution it counts all the times a given method was being called and shows you the percent of time spent in each method. This is a very low overhead method to looking into execution time. Ruby 2.1+ has this available in gem form it's called stackprof. As you guessed you can run this with derailed benchmarks, first add it to your gemfile gem "stackprof", group: :development then execute:

$ bundle exec derailed exec perf:stackprof
==================================
  Mode: cpu(1000)
  Samples: 16067 (1.07% miss rate)
  GC: 2651 (16.50%)
==================================
     TOTAL    (pct)     SAMPLES    (pct)     FRAME
      1293   (8.0%)        1293   (8.0%)     block in ActionDispatch::Journey::Formatter#missing_keys
       872   (5.4%)         872   (5.4%)     block in ActiveSupport::Inflector#apply_inflections
       935   (5.8%)         802   (5.0%)     ActiveSupport::SafeBuffer#safe_concat
       688   (4.3%)         688   (4.3%)     Temple::Utils#escape_html
       578   (3.6%)         578   (3.6%)     ActiveRecord::Attribute#initialize
      3541  (22.0%)         401   (2.5%)     ActionDispatch::Routing::RouteSet#url_for
       346   (2.2%)         346   (2.2%)     ActiveSupport::SafeBuffer#initialize
       298   (1.9%)         298   (1.9%)     ThreadSafe::NonConcurrentCacheBackend#[]
       227   (1.4%)         227   (1.4%)     block in ActiveRecord::ConnectionAdapters::PostgreSQLAdapter#exec_no_cache
       218   (1.4%)         218   (1.4%)     NewRelic::Agent::Instrumentation::Event#initialize
      1102   (6.9%)         213   (1.3%)     ActiveSupport::Inflector#apply_inflections
       193   (1.2%)         193   (1.2%)     ActionDispatch::Routing::RouteSet::NamedRouteCollection::UrlHelper#deprecate_string_options
       173   (1.1%)         173   (1.1%)     ActiveSupport::SafeBuffer#html_safe?
       308   (1.9%)         171   (1.1%)     NewRelic::Agent::Instrumentation::ActionViewSubscriber::RenderEvent#metric_name
       159   (1.0%)         159   (1.0%)     block in ActiveRecord::Result#hash_rows
       358   (2.2%)         153   (1.0%)     ActionDispatch::Routing::RouteSet::Generator#initialize
       153   (1.0%)         153   (1.0%)     ActiveRecord::Type::String#cast_value
       192   (1.2%)         143   (0.9%)     ActionController::UrlFor#url_options
       808   (5.0%)         127   (0.8%)     ActiveRecord::LazyAttributeHash#[]
       121   (0.8%)         121   (0.8%)     PG::Result#values
       120   (0.7%)         120   (0.7%)     ActionDispatch::Journey::Router::Utils::UriEncoder#escape
      2478  (15.4%)         117   (0.7%)     ActionDispatch::Journey::Formatter#generate
       115   (0.7%)         115   (0.7%)     NewRelic::Agent::Instrumentation::EventedSubscriber#event_stack
       114   (0.7%)         114   (0.7%)     ActiveRecord::Core#init_internals
       263   (1.6%)         110   (0.7%)     ActiveRecord::Type::Value#type_cast
      8520  (53.0%)         102   (0.6%)     ActionView::CompiledTemplates#_app_views_repos__repo_html_slim__2939326833298152184_70365772737940

From here you can dig into individual methods.

Is this perf change faster?

Micro benchmarks might tell you at the code level how much faster something is, but what about the overall application speed. If you're trying to figure out how effective a performance change is to your application, it is useful to compare it to your existing app performance. To help you with that you can use:

$ bundle exec derailed exec perf:ips
Endpoint: "/"
Calculating -------------------------------------
                 ips     1.000  i/100ms
-------------------------------------------------
                 ips      3.306  (± 0.0%) i/s -     17.000

This will hit an endpoint in your application using benchmark-ips. In "iterations per second" a higher value is always better. You can run your code change several times using this method, and then run your "baseline" codebase (without your changes) to see how it affects your overall performance. You'll want to run and record the results several times (including the std deviation) so you can help eliminate noise. Benchmarking is hard, this technique isn't perfect but it's definitely better than nothing.

If you care you can also run pure benchmark (without ips):

$ bundle exec derailed exec perf:test

But I wouldn't, benchmark-ips is a better measure.

Configuring benchmark-ips

The benchmark-ips gem allows for a number of test run customizations, and derailed_benchmarks exposes a few of them via environment variables.

• IPS_WARMUP: number of seconds spent warming up the app, defaullt is 2
• IPS_TIME: number of seconds to run ips benchmark for after warm up, defaullt is 5
• IPS_SUITE: custom suite to use to run test
• IPS_ITERATIONS: number of times to run the test, displaying that last result, defaullt is 1

I made a patch to to Rails how can I tell if it made my Rails app faster and test for statistical significance

When you're trying to submit a performance patch to rails/rails then they'll likely ask you for a benchmark. While you can sometimes provide a microbenchmark, a real world full stack request/response test is the gold standard.

That's what this section is about. You'll need a rails app, ideally one you can open source (see example apps if you need inspiration for extracting your private code into something external).

Then you'll need to fork rails and make a branch. Then point your rails app to your branch in your gemfile

gem 'rails', github: "<github username>/rails", branch: "<your branch name>"

or point it at your local copy:

gem 'rails', path: "<path/to/your/local/copy/rails>"

To run your tests within the context of your current app/repo:

$ bundle exec derailed exec perf:app

This will automatically test the two latest commits of your library/current directory.

If you'd like to test the Rails library instead, make sure that ENV[DERAILED_PATH_TO_LIBRARY] is unset.

$ bundle exec derailed exec perf:library

This will automatically test the two latest commits of Rails.

If you would also like to compare against different SHAs you can manually specify them:

$ SHAS_TO_TEST="7b4d80cb373e,13d6aa3a7b70" bundle exec derailed exec perf:library

Use a comma to seperate your branch names with the SHAS_TO_TEST env var, or omit the env var to use the last 2 git commits.

If you only include one SHA, then derailed will grab the latest commit and compare it to that SHA.

These tests might take a along time to run so the output is stored on disk incase you want to see them in the future, they're at tmp/compare_branches/<timestamp> and labeled with the same names as your commits.

When the test is done it will output which commit "won" and by how much:

❤️ ❤️ ❤️  (Statistically Significant) ❤️ ❤️ ❤️

[f1ab117] (11.3844 seconds) "I am the new commit" ref: "winner"
  FASTER 🚀🚀🚀 by:
    1.0062x [older/newer]
    0.6147% [(older - newer) / older * 100]
[5594a2d] (11.4548 seconds) "Old commit" ref: "loser"

Iterations per sample:
Samples: 100

Test type: Kolmogorov Smirnov
Confidence level: 99.0 %
Is significant? (max > critical): true
D critical: 0.2145966026289347
D max: 0.26

Histograms (time ranges are in seconds):

   [f1ab117] description:                                        [5594a2d] description:
     "I am the new commit"                                         "Old commit"
                  ┌                                        ┐                    ┌                                        ┐
   [11.2 , 11.28) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 12                             [11.2 , 11.28) ┤▇▇▇▇ 3
   [11.28, 11.36) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 22                 [11.28, 11.36) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 19
   [11.35, 11.43) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 30       [11.35, 11.43) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 17
   [11.43, 11.51) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 17                       [11.43, 11.51) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 25
   [11.5 , 11.58) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 13                           [11.5 , 11.58) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 15
   [11.58, 11.66) ┤▇▇▇▇▇▇▇ 6                                     [11.58, 11.66) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 13
   [11.65, 11.73) ┤ 0                                            [11.65, 11.73) ┤▇▇▇▇ 3
   [11.73, 11.81) ┤ 0                                            [11.73, 11.81) ┤▇▇▇▇ 3
   [11.8 , 11.88) ┤ 0                                            [11.8 , 11.88) ┤▇▇▇ 2
                  └                                        ┘                    └                                        ┘
                             # of runs in range                                            # of runs in range

You can provide this to the Rails team along with the example app you used to benchmark (so they can independently verify if needed).

Generally performance patches have to be weighted in terms of how much they help versus how large/difficult/gnarly the patch is. If the above example was a really tiny patch and it was in a common component, then half a percent might be a justafiable increase. If it was a huge re-write then it's likely going to be closed. In general I tend to not submit patches unless I'm seeing >= 1% performance increases.

You can use this to test changes in other libraries that aren't rails, you just have to tell it the path to the library you want to test against with the DERAILED_PATH_TO_LIBRARY env var.

To get the best results before running tests you should close all programs on your laptop, turn on a program to prevent your laptop from going to sleep (or increase your sleep timer). Make sure it's plugged into a power outlet and go grab a cup of coffee. If you do anything on your laptop while this test is running you risk the chance of skewing your results.

As the test is executing, intermediate results will be printed every 50 iterations.

Environment Variables

All the tasks accept configuration in the form of environment variables.

Increasing or decreasing test count TEST_COUNT

For tasks that are run a number of times you can set the number using TEST_COUNT for example:

$ TEST_COUNT=100_000 bundle exec derailed exec perf:test

Warming up your app before measuring with WARM_COUNT

When you are measuring the long term performance of an application, especially if you're using jit you may want to let the application "warm up" without measuring this time. To allow for this you can specify WARM_COUNT and the application will be called that number of times before any measurements are taken.

$ WARM_COUNT=5_000 bundle exec derailed exec perf:test
Warming up app: 5000 times
# ...

Hitting a different endpoint with PATH_TO_HIT

By default tasks will hit your homepage /. If you want to hit a different url use PATH_TO_HIT for example if you wanted to go to users/new you can execute:

$ PATH_TO_HIT=/users/new bundle exec derailed exec perf:mem

This method accepts a full uri. For example, allowing you to hit a subdomain endpoint:

$ PATH_TO_HIT=http://subdomain.lvh.me:3000/users/new bundle exec derailed exec perf:mem

Beware that you cannot combine a full uri with USE_SERVER.

Setting HTTP headers

You can specify HTTP headers by setting HTTP_<header name> variables. Example:

$ HTTP_AUTHORIZATION="Basic YWRtaW46c2VjcmV0\n" \
  HTTP_USER_AGENT="Mozilla/5.0" \
  PATH_TO_HIT=/foo_secret bundle exec derailed exec perf:ips

Using a real web server with USE_SERVER

All tests are run without a webserver (directly using Rack::Mock by default), if you want to use a webserver set USE_SERVER to a Rack::Server compliant server, such as webrick.

$ USE_SERVER=webrick bundle exec derailed exec perf:mem

Or

$ USE_SERVER=puma bundle exec derailed exec perf:mem

This boots a webserver and hits it using curl instead of in memory. This is useful if you think the performance issue is related to your webserver.

Note: this plugs in the given webserver directly into rack, it doesn't use any puma.config file etc. that you have set-up. If you want to do this, i'm open to suggestions on how (and pull requests)

Excluding ActiveRecord

By default, derailed will load ActiveRecord if the gem is included as a dependency. It is included by default, if you just include the rails gem. If you are using a different ORM, you will either need to only include the railties gem, or set the DERAILED_SKIP_ACTIVE_RECORD environment variable.

$ DERAILED_SKIP_ACTIVE_RECORD=true

Alternatively, use the DERAILED_SKIP_RAILS_REQUIRES environment variable to have derailed not require any Rails gems. Your app will then need to require them as part of its boot sequence.

Running in a different environment

Tests run against the production environment by default, but it's easy to change this if your app doesn't run locally with RAILS_ENV set to production. For example:

$ RAILS_ENV=development bundle exec derailed exec perf:mem

perf.rake

If you want to customize derailed, you'll need to create a perf.rake file at the root of the directory you're trying to benchmark.

It is possible to run benchmarks directly using rake

$ cat <<  EOF > perf.rake
  require 'bundler'
  Bundler.setup

  require 'derailed_benchmarks'
  require 'derailed_benchmarks/tasks'
EOF

The file should look like this:

$ cat perf.rake
  require 'bundler'
  Bundler.setup

  require 'derailed_benchmarks'
  require 'derailed_benchmarks/tasks'

This is done so the benchmarks will be loaded before your application, this is important for some benchmarks and less for others. This also prevents you from accidentally loading these benchmarks when you don't need them.

Then you can execute your commands via rake.

To find out the tasks available you can use $ rake -f perf.rake -T which essentially says use the file perf.rake and list all the tasks.

$ rake -f perf.rake -T

Rack Setup

Using Rails? You don't need to do anything special. If you're using Rack, you need to tell us how to boot your app. In your perf.rake file add a task:

namespace :perf do
  task :rack_load do
    DERAILED_APP = # your code here
  end
end

Set the constant DERAILED_APP to your Rack app. See schneems/derailed_benchmarks#1 for more info.

An example of setting this up could look like:

# perf.rake

require 'bundler'
Bundler.setup

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

namespace :perf do
  task :rack_load do
    require_relative 'lib/application'
    DERAILED_APP = MyApplication::Routes
  end
end

Authentication

If you're trying to test an endpoint that has authentication you'll need to tell your task how to bypass that authentication. Authentication is controlled by the DerailedBenchmarks.auth object. There is a built in support for Devise. If you're using some other authentication method, you can write your own authentication strategy.

To enable authentication in a test run with:

$ USE_AUTH=true bundle exec derailed exec perf:mem

See below how to customize authentication.

Authentication with Devise

If you're using devise, there is a built in auth helper that will detect the presence of the devise gem and load automatically.

Create a perf.rake file at your root.

$ cat perf.rake

If you want you can customize the user that is logged in by setting that value in your perf.rake file.

DerailedBenchmarks.auth.user = -> { User.find_or_create!(twitter: "schneems") }

You will need to provide a valid user, so depending on the validations you have in your user.rb, you may need to provide different parameters.

If you're trying to authenticate a non-user model, you'll need to write your own custom auth strategy.

Custom Authentication Strategy

To implement your own authentication strategy You will need to create a class that inherits from auth_helper.rb. You will need to implement a setup and a call method. You can see an example of how the devise auth helper was written and how it can be done for Clearance. You can put this code in your perf.rake file.

class MyCustomAuth < DerailedBenchmarks::AuthHelper
  def setup
    # initialize code here
  end

  def call(env)
    # log something in on each request
    app.call(env)
  end
end

The devise strategy works by enabling test mode inside of the Rack request and inserting a stub user. You'll need to duplicate that logic for your own authentication scheme if you're not using devise.

Once you have your class, you'll need to set DerailedBenchmarks.auth to a new instance of your class. In your perf.rake file add:

DerailedBenchmarks.auth = MyCustomAuth.new

Now on every request that is made with the USE_AUTH environment variable set, the MyCustomAuth#call method will be invoked.

License

MIT

Acknowledgements

Most of the commands are wrappers around other libraries, go check them out. Also thanks to @tenderlove as I cribbed some of the Rails init code in $ rake perf:setup from one of his projects.

kthksbye @schneems

derailed_benchmarks's Issues

Multiple devise Models

Hi, I have an app which was 3 authenticable Models
User, Agency, AdminUser

I'm failing to perform the benchmarks.
when calling RAILS_ENV=development USE_AUTH=true bundle exec derailed exec perf:mem_over_time

Booting: development
Endpoint: "/"
Auth: true
/Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
/Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here
PID: 17858
160.21875
/Users/benja/dev/ibilbidea/perf.rake:15:in `call': undefined local variable or method `current_user' for #<MyCustomAuth:0x007fc159f4c028> (NameError)
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/rack-2.0.1/lib/rack/mock.rb:74:in `request'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/rack-2.0.1/lib/rack/mock.rb:56:in `get'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:91:in `call_app'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:174:in `block (3 levels) in <top (required)>'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:173:in `times'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:173:in `block (2 levels) in <top (required)>'
	from /Users/benja/.rvm/gems/ruby-2.4.0@global/gems/rake-12.0.0/lib/rake/task.rb:250:in `block in execute'
	from /Users/benja/.rvm/gems/ruby-2.4.0@global/gems/rake-12.0.0/lib/rake/task.rb:250:in `each'
	from /Users/benja/.rvm/gems/ruby-2.4.0@global/gems/rake-12.0.0/lib/rake/task.rb:250:in `execute'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/airbrake-6.0.0/lib/airbrake/rake/task_ext.rb:19:in `execute'
	from /Users/benja/.rvm/gems/ruby-2.4.0@global/gems/rake-12.0.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'
	from /Users/benja/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
	from /Users/benja/.rvm/gems/ruby-2.4.0@global/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
	from /Users/benja/.rvm/gems/ruby-2.4.0@global/gems/rake-12.0.0/lib/rake/task.rb:180:in `invoke'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/bin/derailed:41:in `exec'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/gems/derailed_benchmarks-1.3.2/bin/derailed:92:in `<top (required)>'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/bin/derailed:22:in `load'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/bin/derailed:22:in `<main>'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/bin/ruby_executable_hooks:15:in `eval'
	from /Users/benja/.rvm/gems/ruby-2.4.0@rails_5/bin/ruby_executable_hooks:15:in `<main>'

I tried this following perf.rake

class MyCustomAuth < DerailedBenchmarks::AuthHelper
  def setup
    # initialize code here
    require 'devise'
    require 'warden'
    extend ::Warden::Test::Helpers
    extend ::Devise::TestHelpers
    Warden.test_mode!
  end

  def call(env)
    # log something in on each request
    login_as(User.last, scope: :user)
    login_as(Agency.last, scope: :agency)
    login_as(AdminUser.last, scope: :admin_user)
    app.call(env)
  end
end

DerailedBenchmarks.auth = MyCustomAuth.new

what I'm missing?

best regards

derailed bundle:mem can't load warden on MRI2.3.3?

Hiya. I'm really stumbling through this trying to get anything derailed does to work. I'm trying to debug a Sinatra application, and aside from not getting DERAILED_APP to work at all, I can't even get bundle exec derailed bundle:mem to work to benchmark gems. I have derailed in my Gemfile, and I've run bundle install but it dies immediately trying to load warden.

bundle exec derailed bundle:mem

bundler: failed to load command: derailed (/Users/shank/.rbenv/versions/2.3.3/bin/derailed)
LoadError: cannot load such file -- warden
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/sinatra_more-0.3.43/lib/sinatra_more/warden_plugin.rb:2:in `<top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/sinatra_more-0.3.43/lib/sinatra_more.rb:5:in `<top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/runtime.rb:91:in `block (2 levels) in require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/runtime.rb:86:in `each'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/runtime.rb:86:in `block in require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/runtime.rb:75:in `each'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler/runtime.rb:75:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/bundler-1.13.6/lib/bundler.rb:106:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/bin/derailed:67:in `block in <class:DerailedBenchmarkCLI>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/bin/derailed:92:in `<top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/bin/derailed:22:in `load'
  /Users/shank/.rbenv/versions/2.3.3/bin/derailed:22:in `<top (required)>'

What am I doing wrong?

Uninitialized constant User when customizing user log in

When following the instructions in the readme to supply a Devise user to authenticate I get an uninitialized constant error since the model is not loaded yet.

# broken perf.rake
require 'bundler'
Bundler.setup

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

DerailedBenchmarks.auth.user = User.find_by_email!("[email protected]")

Monkey-patching DerailedBenchmarks::AuthHelpers::Devise using this syntax works since it defers the constant check until the app is loaded.

# working perf.rake
require 'bundler'
Bundler.setup

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

module DerailedBenchmarks
  def auth.user
    @user ||= User.find_by_email!("[email protected]")
  end
end

Am I missing something with the documented syntax?

If not, I can make a PR that accepts a callable for DerailedBenchmarks.auth.user.

DerailedBenchmarks.auth.user = -> { User.find_by_email!("[email protected]") }

gem not working with Mongoid

I dont understand:

I can start my app in production via

• RAILS_ENV=production bundle exec rails console
• RAILS_ENV=production bundle exec rails server

locally without any problems.

But when I try derailed:

• RAILS_ENV=production derailed exec perf:allocated_objects
  or
• RAILS_ENV=production bundle exec derailed exec perf:allocated_objects

I get this error and I dont know how to solve it

git:(master) ✗ RAILS_ENV=production bundle exec derailed exec perf:allocated_objects
Booting: production
[tunemygc] not enabled
bundler: failed to load command: derailed (/Users/jan/.rbenv/versions/2.4.1/bin/derailed)
LoadError: Could not load 'active_record/connection_adapters/mongodb_adapter'. Make sure that the adapter in config/database.yml is valid. If you use an adapter other than 'mysql2', 'postgresql' or 'sqlite3' add the necessary adapter gem to the Gemfile.
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/dependencies.rb:293:in `require'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/dependencies.rb:293:in `block in require'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/dependencies.rb:259:in `load_dependency'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/dependencies.rb:293:in `require'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.2/lib/active_record/connection_adapters/connection_specification.rb:174:in `spec'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.2/lib/active_record/connection_handling.rb:53:in `establish_connection'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.2/lib/active_record/railtie.rb:125:in `block (2 levels) in <class:Railtie>'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/lazy_load_hooks.rb:43:in `instance_eval'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/lazy_load_hooks.rb:43:in `execute_hook'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/lazy_load_hooks.rb:33:in `block in on_load'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/lazy_load_hooks.rb:32:in `each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activesupport-5.0.2/lib/active_support/lazy_load_hooks.rb:32:in `on_load'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/activerecord-5.0.2/lib/active_record/railtie.rb:121:in `block in <class:Railtie>'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/railties-5.0.2/lib/rails/initializable.rb:30:in `instance_exec'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/railties-5.0.2/lib/rails/initializable.rb:30:in `run'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/railties-5.0.2/lib/rails/initializable.rb:55:in `block in run_initializers'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:228:in `block in tsort_each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:350:in `block (2 levels) in each_strongly_connected_component'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:431:in `each_strongly_connected_component_from'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:349:in `block in each_strongly_connected_component'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:347:in `each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:347:in `call'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:347:in `each_strongly_connected_component'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:226:in `tsort_each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/tsort.rb:205:in `tsort_each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/railties-5.0.2/lib/rails/initializable.rb:54:in `run_initializers'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/railties-5.0.2/lib/rails/application.rb:352:in `initialize!'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:26:in `block (2 levels) in <top (required)>'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `block in execute'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `execute'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:180:in `invoke'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:60:in `block (2 levels) in <top (required)>'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `block in execute'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `execute'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:216:in `block in invoke_prerequisites'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:214:in `each'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:214:in `invoke_prerequisites'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:193:in `block in invoke_with_call_chain'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:180:in `invoke'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/derailed_benchmarks-1.3.2/bin/derailed:41:in `exec'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
  /Users/jan/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/derailed_benchmarks-1.3.2/bin/derailed:92:in `<top (required)>'
  /Users/jan/.rbenv/versions/2.4.1/bin/derailed:22:in `load'
  /Users/jan/.rbenv/versions/2.4.1/bin/derailed:22:in `<top (required)>'

Warning before benchmarks

This isn't a problem. Whenever I run the command derailed bundle:mem, for example, a warning is displayed:

WARN: Unresolved specs during Gem::Specification.reset:
      rack (~> 1)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.

However, running the same command with bundle exec, this warning is no longer displayed. I understand what it is. If you think this relevant, we can adjust it.

Thanks, Richard.
Best!

Task 'setup' with USE_SERVER not failing with a 302 response

I've noticed that this line seems to ignore a 302 response - is this the desired behaviour? This happens, for example, when authentication fails with a controller redirect (am learning this the hard way trying to write custom AuthHelper for my Rack app). In this case I'm assuming the generated output is from the redirect path & not the intended PATH_TO_HIT. If so this seems misleading.

Suggestion:
Curl can return response codes, so could this be used to check for a 200 response - i.e. same logic as when USE_SERVER is not set?

PATH_TO_HIT shows error

Ruby 2.2.2 and Rails 3.2.22
I am following this link https://github.com/schneems/derailed_benchmarks, and running this server in production mode, when i was running this
PATH_TO_HIT=/setup_others/new_reason bundle exec derailed exec perf:mem

Got this error
Booting: production
Endpoint: "/setup_others/new_reason"
/Users/test/.rvm/gems/ruby-2.2.2/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
/Users/test/.rvm/gems/ruby-2.2.2/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here Impact of require <file> on RAM

Showing all require <file> calls that consume 0.3 MiB or more of RSS
Configure with CUT_OFF=0 for all entries or CUT_OFF=5 for few entries
Note: Files only count against RAM on their first load.
If multiple libraries require the same file, then
the 'cost' only shows up under the first library

/Users/test/.rvm/gems/ruby-2.2.2/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:92:in `call_app': Bad request: You are being redirected. (RuntimeError)

Getting NoMethodError: undefined method `initialized?' error when running perf:require_bench

I'm running a Rails 3.2.19 app and when I run the perf:require_benchmarks I get the following. It works fine on a Rails 4 app though:

neel@Neels-MacBook-Pro:/railsapps/imua$ bundle exec rake -f perf.rake perf:require_bench --trace
Your Gemfile lists the gem rspec-rails (> 3.0.0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
Your Gemfile lists the gem factory_girl_rails (~> 4.0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
Your Gemfile lists the gem jasmine (>= 0) more than once.
You should probably keep only one of them.
While it's not a problem now, it could cause errors if you change the version of just one of them later.
** Invoke perf:require_bench (first_time)
** Invoke perf:kernel_require_patch (first_time)
** Execute perf:kernel_require_patch
** Invoke perf:setup (first_time)
** Execute perf:setup
rake aborted!
NoMethodError: undefined method initialized?' for #Imua::Application:0x007fe5a2bf4e30
/Users/neel/.rvm/gems/ruby-2.1.2/gems/derailed_benchmarks-0.0.0/lib/derailed_benchmarks/tasks.rb:29:inblock (2 levels) in '
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:240:in call'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:240:inblock in execute'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:235:in each'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:235:inexecute'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:179:in block in invoke_with_call_chain'
/Users/neel/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/monitor.rb:211:inmon_synchronize'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:172:in invoke_with_call_chain'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:201:inblock in invoke_prerequisites'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:199:in each'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:199:ininvoke_prerequisites'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:178:in block in invoke_with_call_chain'
/Users/neel/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/monitor.rb:211:inmon_synchronize'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:172:in invoke_with_call_chain'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/task.rb:165:ininvoke'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:150:in invoke_task'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:106:inblock (2 levels) in top_level'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:106:in each'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:106:inblock in top_level'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:115:in run_with_threads'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:100:intop_level'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:78:in block in run'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:176:instandard_exception_handling'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/lib/rake/application.rb:75:in run'
/Users/neel/.rvm/gems/ruby-2.1.2/gems/rake-10.3.2/bin/rake:33:in'
/Users/neel/.rvm/gems/ruby-2.1.2/bin/rake:23:in load'
/Users/neel/.rvm/gems/ruby-2.1.2/bin/rake:23:in'
Tasks: TOP => perf:require_bench => perf:setup

I'm really excited to use this library but hopefully it's compatible with Rails 3.2.19..

NoMethodError: undefined method `locked_gems' for Bundler:Module

$ rake -f perf.rake perf:require_bench
rake aborted!
NoMethodError: undefined method `locked_gems' for Bundler:Module
/Users/Ackerman/.rvm/gems/ruby-2.0.0-p353/bundler/gems/derailed_benchmarks-36fd32239808/lib/derailed_benchmarks.rb:10:in `gem_is_bundled?'
/Users/Ackerman/.rvm/gems/ruby-2.0.0-p353/bundler/gems/derailed_benchmarks-36fd32239808/lib/derailed_benchmarks/tasks.rb:53:in `block (2 levels) in <top (required)>'
/Users/Ackerman/.rvm/gems/ruby-2.0.0-p353/bin/ruby_executable_hooks:15:in `eval'
/Users/Ackerman/.rvm/gems/ruby-2.0.0-p353/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => perf:require_bench => perf:setup
(See full trace by running task with --trace)

Any idea?

airbrake causing problems

Awesome project!

I'm trying to hunt down memory leaks, and I had trouble starting perf:mem_over_time until I added various Airbrake keys, but then I'm still getting an error (testing our Rails app)

→ AIRBRAKE_PROJECT_ID=6 AIRBRAKE_API_KEY=XXXX bundle exec derailed exec perf:mem_over_time
Booting: production
Endpoint: "/"
/Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
/Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here
PID: 79590
211.39453125
/Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:92:in `call_app': Bad request:  (RuntimeError)
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:174:in `block (3 levels) in <top (required)>'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:173:in `times'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:173:in `block (2 levels) in <top (required)>'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/rake-12.1.0/lib/rake/task.rb:251:in `block in execute'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/rake-12.1.0/lib/rake/task.rb:251:in `each'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/rake-12.1.0/lib/rake/task.rb:251:in `execute'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/airbrake-6.0.0/lib/airbrake/rake/task_ext.rb:19:in `execute'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/rake-12.1.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'
	from /Users/tansaku/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/rake-12.1.0/lib/rake/task.rb:188:in `invoke_with_call_chain'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/rake-12.1.0/lib/rake/task.rb:181:in `invoke'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/bin/derailed:41:in `exec'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/bin/derailed:92:in `<top (required)>'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/bin/derailed:23:in `load'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/bin/derailed:23:in `<main>'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
	from /Users/tansaku/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'

The full project is https://github.com/AgileVentures/WebsiteOne/ - maybe I just turn off Airbrake - or might it be something else ...?

Benchmarking Sinatra?

Okay, so I read #84 and the readme about how to load an arbitrary Rack app. However, I don't know much about Rack + Sinatra modular apps, or at least, enough on how to load them in derailed.

I'm trying to run bundle exec derailed exec perf:mem_over_time with the following perf.rake:

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

namespace :perf do
  task :rack_load do

    require_relative './Jump'
    DERAILED_APP = Jump.run!
  end
end

where Jump is a modular Sinatra, app, e.g.,

require 'sinatra'
require 'newrelic_rpm'
require 'sinatra/partial'
require 'sinatra_more/markup_plugin'
require 'mongoid'
require 'bcrypt'
require 'raven'
require 'skylight/sinatra'
require 'redis'
require 'sidekiq'

class Jump < Sinatra::Base
# snip
end

and where the config.ru is

require './Jump'
run Jump

If I start it, this happens:

$ bundle exec derailed exec perf:mem_over_time

You're not using Rails
You need to tell derailed how to boot your app
In your perf.rake add:

namespace :perf do
  task :rack_load do
    # DERAILED_APP = your code here
  end
end
Jump development server is online.
== Sinatra (v1.4.7) has taken the stage on 4567 for development with backup from Puma
Puma starting in single mode...
* Version 3.6.2 (ruby 2.3.3-p222), codename: Sleepy Sunday Serenity
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://localhost:4567

and nothing else unless I hit CTRL + C, at which point it shuts down and I end up with

== Sinatra has ended his set (crowd applauds)
- Gracefully stopping, waiting for requests to finish
=== puma shutdown: 2017-01-09 20:21:17 -0700 ===
- Goodbye!
Endpoint: "/"
bundler: failed to load command: derailed (/Users/shank/.rbenv/versions/2.3.3/bin/derailed)
LoadError: cannot load such file -- rack/test
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/skylight-1.0.1/lib/skylight/probes.rb:81:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/skylight-1.0.1/lib/skylight/probes.rb:81:in `require'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:69:in `block (2 levels) in <top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:248:in `block in execute'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:in `each'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:243:in `execute'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:187:in `block in invoke_with_call_chain'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:180:in `invoke_with_call_chain'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:209:in `block in invoke_prerequisites'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:207:in `each'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:207:in `invoke_prerequisites'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:186:in `block in invoke_with_call_chain'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:180:in `invoke_with_call_chain'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-11.3.0/lib/rake/task.rb:173:in `invoke'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/bin/derailed:41:in `exec'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
  /Users/shank/.rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.1/bin/derailed:92:in `<top (required)>'
  /Users/shank/.rbenv/versions/2.3.3/bin/derailed:22:in `load'
  /Users/shank/.rbenv/versions/2.3.3/bin/derailed:22:in `<top (required)>'

How exactly do I properly mount a modular Sinatra app using derailed?

Memory leak in an empty rails app?

I've got a production rails app which has been having issues as far as the server running out of memory consistently over time. I came across your gem and thought I'd try it out, and sure enough just hitting an endpoint, I consistently saw the memory output in the file being written to tmp going up and up...

However, I wanted to prove that this was in fact demonstrating the problem, so I created a new rails app (4.1.4) with ruby 2.2.5, made a single controller with an action that does:

def index
  head 200
end

When I tried hitting that endpoint with bundle exec derailed exec perf:mem_over_time, the results I get are:

https://gist.github.com/patrick99e99/1fcb853bbc3fdc939053a6818462f78b

which does basically the same thing.. the number goes up and up and up.................. So......... I am a little confused? Does a brand new rails app that does nothing have a memory leak? Or am I misreading these numbers and this i actually not a problem?

Command not found: derailed

The Gemfile:
https://gist.github.com/chrisccerami/dd5233cfccd67a2a4505

Bundle list:
Gems included by the bundle:

• actionmailer (4.0.0)
• actionpack (4.0.0)
• activeadmin (0.6.1 1f4f90c)
• activemodel (4.0.0)
• activerecord (4.0.0)
• activerecord-deprecated_finders (1.0.3)
• activesupport (4.0.0)
• addressable (2.3.5)
• akismetor (1.0.0)
• arbre (1.0.1)
• arel (4.0.0)
• atomic (1.1.13)
• axiom-types (0.0.5)
• bcrypt-ruby (3.1.2)
• benchmark-ips (2.2.0)
• better_errors (1.0.1)
• binding_of_caller (0.7.2)
• bourbon (3.1.8)
• browser (0.8.0)
• builder (3.1.4)
• bundler (1.7.7)
• capybara (2.1.0)
• capybara-webkit (1.0.0)
• celluloid (0.15.2)
• ckeditor (4.0.6)
• climate_control (0.0.3)
• cocaine (0.5.1)
• coderay (1.0.9)
• coercible (1.0.0)
• coffee-rails (4.0.0)
• coffee-script (2.2.0)
• coffee-script-source (1.6.3)
• debug_inspector (0.0.2)
• derailed (0.1.0)
• derailed_benchmarks (0.0.0)
• descendants_tracker (0.0.3)
• devise (3.1.0)
• diff-lcs (1.2.4)
• dotenv (1.0.2)
• dotenv-rails (1.0.2)
• equalizer (0.0.9)
• erubis (2.7.0)
• excon (0.25.3)
• execjs (2.0.0)
• factory_girl (4.2.0)
• factory_girl_rails (4.2.1)
• faraday (0.8.8)
• ffi (1.9.3)
• fog (1.15.0)
• formatador (0.2.4)
• formtastic (2.3.0.rc2)
• geocoder (1.1.8)
• get_process_mem (0.2.0)
• guard (2.2.5)
• guard-rspec (4.2.3)
• has_scope (0.6.0.rc)
• hashie (2.0.5)
• hike (1.2.3)
• httpauth (0.2.0)
• i18n (0.6.5)
• ice_nine (0.11.0)
• inherited_resources (1.4.1)
• jquery-rails (3.0.4)
• jquery-ui-rails (4.0.5)
• json (1.8.0)
• jwt (0.1.8)
• kaminari (0.14.1)
• kgio (2.8.1)
• launchy (2.3.0)
• listen (2.4.0)
• lumberjack (1.0.4)
• mail (2.5.4)
• memory_profiler (0.9.0)
• method_source (0.8.2)
• mime-types (1.24)
• mini_portile (0.5.1)
• minitest (4.7.5)
• multi_json (1.7.9)
• multipart-post (1.2.0)
• net-scp (1.1.2)
• net-ssh (2.7.0)
• newrelic_rpm (3.6.9.171)
• nokogiri (1.6.0)
• oauth (0.4.7)
• oauth2 (0.8.1)
• omniauth (1.1.4)
• omniauth-facebook (1.4.1)
• omniauth-oauth (1.0.1)
• omniauth-oauth2 (1.1.1)
• omniauth-twitter (1.0.0)
• orm_adapter (0.4.0)
• paperclip (3.5.1)
• pg (0.16.0)
• polyamorous (0.6.4)
• polyglot (0.3.3)
• pry (0.9.12.2)
• pry-nav (0.2.3)
• quiet_assets (1.0.2)
• rack (1.5.2)
• rack-test (0.6.2)
• rails (4.0.0)
• rails_12factor (0.0.2)
• rails_serve_static_assets (0.0.1)
• rails_stdout_logging (0.0.1)
• railties (4.0.0)
• raindrops (0.12.0)
• rake (10.1.0)
• ransack (1.0.0)
• rb-fsevent (0.9.4)
• rb-inotify (0.9.3)
• rdoc (3.12.2)
• responders (1.0.0)
• rspec (2.14.1)
• rspec-core (2.14.5)
• rspec-expectations (2.14.2)
• rspec-mocks (2.14.3)
• rspec-rails (2.14.0)
• ruby-hmac (0.4.0)
• sass (3.2.10)
• sass-rails (4.0.0)
• sdoc (0.3.20)
• shoulda-matchers (2.3.0)
• simple_form (1.4.1)
• slop (3.4.6)
• sprockets (2.10.0)
• sprockets-rails (2.0.0)
• textacular (3.1.0)
• thor (0.18.1)
• thread_safe (0.1.2)
• tilt (1.4.1)
• timecop (0.6.3)
• timers (1.1.0)
• treetop (1.4.15)
• tzinfo (0.3.37)
• uglifier (2.2.0)
• unicorn (4.7.0)
• virtus (1.0.1)
• warden (1.2.3)
• webshims-rails (1.11.6.1)
• wepay (0.0.3 43a98d4)
• xpath (2.0.0)
• zonebie (0.5.1)

It appears as though derailed_benchmarks (0.0.0) is being installed rather than derailed_benchmarks (1.0.0) like in other apps. When setting derailed_benchmarks explicitly to 1.0.0 in my Gemfile, I received the following:

Bundler could not find compatible versions for gem "rake":
  In Gemfile:
    derailed_benchmarks (~> 1.0.0) ruby depends on
      rake (~> 10.4) ruby

    sass-rails (~> 4.0.0) ruby depends on
      railties (< 5.0, >= 4.0.0.beta) ruby depends on
        rake (10.1.0)

perf:require_bench not working as expected

I just read your great post on http://www.schneems.com/2014/11/07/i-ram-what-i-ram.html , and had to do two tweaks to get bundle exec rake -f perf.rake perf:require_bench to work as expected (using derailed_benchmarks master):

1. perf:require_bench loads derailed_benchmarks/core_ext/kernel_require.rb after the setup task is executed. This way I don't get measures for the requires that are called during bootup. My solution: Require this file in perf.rake.
2. With CUT_OFF != 0, I never got any output since the TOP_REQUIRE node always has a zero cost. My workaround is to check for name != "TOP" in https://github.com/schneems/derailed_benchmarks/blob/master/lib/derailed_benchmarks/require_tree.rb#L37

I am not absolutely sure how to fix those two issues, but I at least wanted to report them as others might stumple upon similar things.

Thank you for your great work!

Error running outside an Rails app

task/perf.rake is filled with:

namespace :perf do                                                                                                                                                                           
  task :rack_load do                                                                                                                                                                         
    MyTask.new.run                                                                                                                      
  end                                                                                                                                                                                        
end   

~/P/o/scheduler ❯❯❯ derailed exec perf:objects                                                                                                                                 ⏎master ✭ ✱ ◼
You're not using Rails
You need to tell derailed how to boot your app
In your perf.rake add:

namespace :perf do
  task :rack_load do
    # DERAILED_APP = your code here
  end
end
Endpoint: "/"
/home/braulio/.rvm/gems/ruby-2.4.2/gems/derailed_benchmarks-1.3.4/lib/derailed_benchmarks/tasks.rb:71:in `block (3 levels) in <top (required)>': undefined method `starts_with?' for "ANT_HOME":String (NoMethodError)
Did you mean?  start_with?
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/derailed_benchmarks-1.3.4/lib/derailed_benchmarks/tasks.rb:71:in `select'                                                              
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/derailed_benchmarks-1.3.4/lib/derailed_benchmarks/tasks.rb:71:in `block (2 levels) in <top (required)>'                                
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:251:in `block in execute'                                                                                 
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:251:in `each'                                                                                             
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:251:in `execute'                                                                                          
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'                                                                  
        from /home/braulio/.rvm/rubies/ruby-2.4.2/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'                                                                                        
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:188:in `invoke_with_call_chain'                                                                           
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:217:in `block in invoke_prerequisites'                                                                    
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:215:in `each'                                                                                             
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:215:in `invoke_prerequisites'                                                                             
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'                                                                  
        from /home/braulio/.rvm/rubies/ruby-2.4.2/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'                                                                                        
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:188:in `invoke_with_call_chain'                                                                           
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/rake-12.3.0/lib/rake/task.rb:181:in `invoke'                                                                                           
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/derailed_benchmarks-1.3.4/bin/derailed:41:in `exec'                                                                                    
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'                                                                                            
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'                                                                             
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'                                                                                              
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/thor-0.20.0/lib/thor/base.rb:466:in `start'                                                                                            
        from /home/braulio/.rvm/gems/ruby-2.4.2/gems/derailed_benchmarks-1.3.4/bin/derailed:92:in `<top (required)>'                                                                        
        from /home/braulio/.rvm/gems/ruby-2.4.2/bin/derailed:23:in `load'
        from /home/braulio/.rvm/gems/ruby-2.4.2/bin/derailed:23:in `<main>'
        from /home/braulio/.rvm/gems/ruby-2.4.2/bin/ruby_executable_hooks:15:in `eval'
        from /home/braulio/.rvm/gems/ruby-2.4.2/bin/ruby_executable_hooks:15:in `<main>'

Duplicated key error on :service_fee when running bundle exec derailed exec

Hello,

When I run the command bundle exec derailed exec perf:heap with USE_SERVER=puma and here is the errors.

Booting: production
/Users/home/Documents/Projects/pistachio/app/models/finance/transactions/name_to_group.rb:4: warning: duplicated key at line 13 ignored: :service_fee
Endpoint: "/"
/Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
/Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here
Running 1000 times
/Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:92:in `call_app': Bad request:  (RuntimeError)
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:275:in `block (3 levels) in <top (required)>'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:274:in `times'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:274:in `block (2 levels) in <top (required)>'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `call'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/bin/derailed:41:in `exec'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
    from /Users/home/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.1/bin/derailed:92:in `<top (required)>'
    from /Users/home/.rbenv/versions/2.2.3/bin/derailed:23:in `load'
    from /Users/home/.rbenv/versions/2.2.3/bin/derailed:23:in `<main>'

NoMethodError: private method `warn' called for nil:NilClass

Following the direction in the README, I installed the gem and ran "bundle exec derailed bundle:mem". I immediately get the following error:

 NoMethodError: private method `warn' called for nil:NilClass

It originates on line 210 in lib/derailed_benchmarks/tasks.rb

task :ram_over_time do
  Kernel.warn("The ram_over_time task is deprecated. Use mem_over_time")
  Rake::Task["perf:ram_over_time"].invoke
end

IPS_WARMUP and IPS_TIME configuration

Hey @schneems thanks for your work on this gem!

Would you accept a PR for being able to specify the Benchmark.ips :config and :time options? As of now the perf:ips task is not useful if the endpoint you are hitting is very slow (over ~1 second) since the default :time of 5 seconds is not enough to eliminate noise. The proposed API would be

IPS_WARMUP=5 IPS_TIME=30 derailed exec perf:ips

derailed exec perf:em_over_time returns NoDatabaseError

I am trying to fix some memory leaks in my app, but when i do a derailed exec perf:mem_over_time it returns ActiveRecord::NoDatabaseError (FATAL: database "myapp_production" does not exist.

What can i do to fix it?

Authentication protected app issue

I have an application where the root path is a page with omniauth-crowd authentication. Because of this, call_app fails with

Bad request: <html><body>You are being <a href="http://example.org/auth/crowd/">redirected</a>.</body></html>

Should it really consider a redirect a failure?

Allocated Memory Changes

Hello,

This code is working perfectly with our application but we are seeing some seemingly alarming results that I wanted to inquire about.

When I run:
bundle exec derailed exec perf:objects
as well as:
TEST_COUNT=10 bundle exec derailed exec perf:objects

I see our allocated memory increase by a multiple of 10. Is this normal behavior?

Here is a sample of the output I see for TEST_COUNT=1:

allocated memory by gem

3604658 haml-4.0.5
1921843 activerecord-4.1.6
1879824 parslet-1.6.0
1749971 activesupport-4.1.6
737334 activejunky2/app
448577 rack-1.5.5
392121 other
196629 arel-5.0.1.20140414130214
169070 newrelic_rpm-3.12.1.298
131902 actionpack-4.1.6
120986 actionview-4.1.6
118832 thread_safe-0.3.5
108375 dalli-2.7.2
96825 pg-0.17.1
49667 2.2.1/lib
17208 warden-1.2.3
15536 activejunky2/lib
13600 active_hash-1.4.0
12976 activemodel-4.1.6
7258 devise-3.4.0
3225 shortcode-1.0.0
2241 railties-4.1.6
2059 omniauth-1.2.2
1993 json-1.8.3
1920 hashie-3.4.3
1688 css_splitter-0.4.1
1040 sprockets-rails-2.3.3
368 tzinfo-1.2.2
288 sentry-raven-0.13.3
240 sunspot_rails-2.1.1
200 sprockets-2.11.0
167 sunspot-2.1.1

and here is for TEST_COUNT=10:

allocated memory by gem

36009140 haml-4.0.5
19218430 activerecord-4.1.6
18794640 parslet-1.6.0
17497676 activesupport-4.1.6
7373340 activejunky2/app
4485770 rack-1.5.5
3921210 other
1966290 arel-5.0.1.20140414130214
1691701 newrelic_rpm-3.12.1.298
1318660 actionpack-4.1.6
1200761 actionview-4.1.6
1188320 thread_safe-0.3.5
1075010 dalli-2.7.2
968250 pg-0.17.1
496670 2.2.1/lib
172080 warden-1.2.3
155360 activejunky2/lib
136000 active_hash-1.4.0
129760 activemodel-4.1.6
72580 devise-3.4.0
32250 shortcode-1.0.0
22410 railties-4.1.6
20590 omniauth-1.2.2
19930 json-1.8.3
19200 hashie-3.4.3
16880 css_splitter-0.4.1
10400 sprockets-rails-2.3.3
3680 tzinfo-1.2.2
2880 sentry-raven-0.13.3
2400 sunspot_rails-2.1.1
2000 sprockets-2.11.0
1670 sunspot-2.1.1

Any feedback you are able to provide would be GREATLY appreciated.

Many thanks,
Ellie

Don't know how to build task 'perf:require_bench'

I did the setup however when I run:
$ rake -f perf.rake perf:require_bench
rake aborted!
Don't know how to build task 'perf:require_bench'
/Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/task_manager.rb:62:in []' /Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:149:ininvoke_task'
/Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:106:in block (2 levels) in top_level' /Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:106:ineach'
/Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:106:in block in top_level' /Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:115:inrun_with_threads'
/Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:100:in top_level' /Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:78:inblock in run'
/Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:176:in standard_exception_handling' /Users/luiszacheu/.rvm/rubies/ruby-2.2.4/lib/ruby/2.2.0/rake/application.rb:75:inrun'
/Users/luiszacheu/.rvm/rubies/ruby-2.2.4/bin/rake:33:in `

'

Returns this trace, do not understand what is happening can help me?
I'm using rails with activeadmin

thank you!

Support for others HTTP Method (POST, PUT, DELETE, etc...)

After reading the source code, I don't think that's it's possible to do a call with anything else than GET.

This feature could be a good idea.
What do you thing?

Bad request error

When running derailed exec perf:mem_over_time I got the following err

PID: 7485
I, [2017-05-22T16:09:00.005495 #7485]  INFO -- : Started GET "/" for  at 2017-05-22 16:09:00 +0545
164.84375
I, [2017-05-22T16:09:00.039285 #7485]  INFO -- : Processing by ProjectsController#index as HTML
I, [2017-05-22T16:09:00.042417 #7485]  INFO -- : Redirected to http://www.example.org/
I, [2017-05-22T16:09:00.042492 #7485]  INFO -- : Filter chain halted as :force_www rendered or redirected
I, [2017-05-22T16:09:00.042615 #7485]  INFO -- : Completed 302 Found in 3ms (ActiveRecord: 0.0ms)
/home/sushant/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:92:in `call_app': Bad request: <html><body>You are being <a href="http://www.example.org/">redirected</a>.</body></html> (RuntimeError)
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:174:in `block (3 levels) in <top (required)>'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:173:in `times'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:173:in `block (2 levels) in <top (required)>'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/rake-12.0.0/lib/rake/task.rb:250:in `block in execute'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/rake-12.0.0/lib/rake/task.rb:250:in `each'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/rake-12.0.0/lib/rake/task.rb:250:in `execute'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/rake-12.0.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'
	from /home/sushant/.rvm/rubies/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/rake-12.0.0/lib/rake/task.rb:180:in `invoke'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/bin/derailed:41:in `exec'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
	from /home/sushant/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.2/bin/derailed:92:in `<top (required)>'
	from /home/sushant/.rvm/gems/ruby-2.3.1/bin/derailed:22:in `load'
	from /home/sushant/.rvm/gems/ruby-2.3.1/bin/derailed:22:in `<main>'
	from /home/sushant/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `eval'
	from /home/sushant/.rvm/gems/ruby-2.3.1/bin/ruby_executable_hooks:15:in `<main>'

Consider adding option to ignore database

Thanks for this awesome gem! I used it to track down a gem that consumed a lot of memory, and submitted a pull request that eliminated the problem.

In a separate app that doesn't use a database, I just tried running derailed exec perf:mem, and it complained that it could not load database configuration. No such file - ["config/database.yml"]. Maybe I missed it, but I couldn't find a way to specify that my app doesn't use a database.

Rails not being loaded in rake task?

I'm using Ruby 2.1.5+Rails 4.2.0 and derailed_benchmarks 1.0.0.
I wanted to do some tests in a Rails app that uses Devise for authenticated the User class.
I tried creating a perf.rake such as the one bellow, with and without the namespace block but I always get the error bellow. It seems that rails is not being load. Guess that I'm missing something simple..?!
Any clue?

require 'bundler'
Bundler.setup

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

namespace :perf do
  task :rack_load do
    DERAILED_APP = Rails.application # your code here
  end
end

DerailedBenchmarks.auth.user = User.find_or_create!(email: "[email protected]")

$  bundle exec rake -f perf.rake perf:ram_over_time --trace
rake aborted!
NameError: uninitialized constant User
/Users/sergio/work/capthost/perf.rake:13:in `<top (required)>'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/rake_module.rb:28:in `load'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/rake_module.rb:28:in `load_rakefile'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:689:in `raw_load_rakefile'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:94:in `block in load_rakefile'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:93:in `load_rakefile'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:77:in `block in run'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:176:in `standard_exception_handling'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/lib/rake/application.rb:75:in `run'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/gems/rake-10.4.2/bin/rake:33:in `<top (required)>'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/bin/rake:23:in `load'
/Users/sergio/work/capthost/vendor/ruby/2.1.0/bin/rake:23:in `<main>'

Subdomain support in `PATH_TO_HIT`?

Thanks for the library - I think this is the best out there for diagnosing memory issues...

Question: running against a real webserver, it looks like adding support for subdomains is possible - as we're just sub-shelling out to curl. But what about Rack::Mock? Is it possible to use subdomains in the endpoint in this case?

Thanks again!

Error when running derailed exec perf:ram_over_time

I'm running the following command:
TEST_COUNT=50_000 derailed exec perf:ram_over_time

The name of my rails app is "job_board"

Which results in the following errors:
booting: production
Digest::Digest is deprecated; use Digest
job_board already exists
Endpoint: "/"
{my-local-path}/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
{my-local-path}/derailed_benchmarks-1.0.1/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here
PID: 11839
152.453125
{my-local-path}/derailed_benchmarks-1.0.1/lib/derailed_benchmarks/tasks.rb:92:in `call_app': Bad request: (RuntimeError)

Any thoughts as to what is happening here?

Add "subdomain" option for mem_over_time task

I am working on diagnosing a memory leak in an app that uses a subdomain architecture. In order to use derailed's mem_over_time task, I needed to manually edit one of the files in the gem that contains a hardcoded reference to http://localhost. It would be nice if there were a SUBDOMAIN option, similar to the PATH_TO_HIT option, that allowed users to optionally supply a subdomain to hit.

Detected parent died, dying

I'm trying to use this gem with puma (3.8.1) web server , but anytime I run one of the tasks, the web server dies on me with this message. I don't know what might be happening. Has anyone seen this type of error before?

Is `task :foo` a relic from when the project was still in development?

Noticed this task while digging through the project (really cool stuff by the way!):

https://github.com/schneems/derailed_benchmarks/blob/2c6fb1d/lib/derailed_benchmarks/tasks.rb#L205-L232

Seems like it was added in your second commit, modified to remove some noise in the commit after, but then really hasn't been touched since. Also seems like it doesn't have any tests/documentation around it, and also isn't a terribly descriptive task name. 😉

Just came across it while reading through the code base and was curious if it got left in by mistake. Should be a simple exercise to remove it if that is the case, just didn't want to do it myself incase I was wrong about it's value (which happens... a lot).

failed to load command: derailed | How can I fix this?

bundler: failed to load command: derailed (/usr/local/opt/rbenv/versions/2.3.3/bin/derailed)
RuntimeError: Bad request:
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:92:in call_app' /usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.2/lib/derailed_benchmarks/tasks.rb:252:in block (2 levels) in <top (required)>'
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in block in execute' /usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in each'
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in execute' /usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:194:in block in invoke_with_call_chain'
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/2.3.0/monitor.rb:214:in mon_synchronize' /usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:187:in invoke_with_call_chain'
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:180:in invoke' /usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.2/bin/derailed:41:in exec'
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/command.rb:27:in run' /usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in invoke_command'
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor.rb:369:in dispatch' /usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/thor-0.19.4/lib/thor/base.rb:444:in start'
/usr/local/opt/rbenv/versions/2.3.3/lib/ruby/gems/2.3.0/gems/derailed_benchmarks-1.3.2/bin/derailed:92:in <top (required)>' /usr/local/opt/rbenv/versions/2.3.3/bin/derailed:23:in load'
/usr/local/opt/rbenv/versions/2.3.3/bin/derailed:23:in `<top (required)>'

how would you go about benchmarking paths that require authentications ?

I'm trying to benchmark some pages that needs the current user to be already logged in. How would I go about this with derailed_benchmarks ?
Thanks

ignoring TEST_COUNT ?

i'm trying to use TEST_COUNT param and it seems to be ending way earlier than 100 or even 1000 tries

TEST_COUNT=1000 USE_SERVER=puma USE_AUTH=true PATH_TO_HIT=/api/v1/notifications.json RAILS_ENV=staging derailed exec perf:mem_over_time
Booting: staging
db_development already exists
Endpoint: "/api/v1/notifications.json"
Auth: true
Port: 3419
Server: "puma"
Puma 2.15.3 starting...
* Min threads: 0, max threads: 16
* Environment: none
* Listening on tcp://0.0.0.0:3419
PID: 52447
196.63671875
203.9296875
204.78515625
205.76953125
206.046875
206.07421875
206.2109375
206.29296875
206.41796875
206.51171875
206.6328125
206.64453125

#kicked out to shell here

there were no errors in my log, just regular hits to the server ( i set log level to debug as well ). are there any other debug flags I could set to try to figre out why its stopping early?

Runtime Error: Bad request

I asked this question on stackoverflow and have not received an answer, so I am raising as an issue here.

I am trying to use [derailed_benchmark to track down a memory leak in a rails app.

When I run the test

USE_SERVER=puma bundle exec derailed exec perf:mem_over_time

the test runs until the memory usage starts to level off, then then the test throws an error:

RuntimeError: Bad request to "curl 'http://localhost:3000/' -s --fail 2>&1" Response:

How do I fix this?

The output is as follows:

perf:mem_over_time
Booting: production
Endpoint: "/"
/Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
/Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here
Port: 3000
Server: "puma"
Puma 2.16.0 starting...
* Min threads: 0, max threads: 16
* Environment: none
* Listening on tcp://0.0.0.0:3000
PID: 17093
125.01171875
128.15234375
132.0546875
133.5078125
133.68359375
133.8828125
bundler: failed to load command: derailed (/Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/bin/derailed)
RuntimeError: Bad request to "curl 'http://localhost:3000/' -s --fail 2>&1" Response:
""
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:85:in `call_app'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:174:in `block (3 levels) in <top (required)>'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:173:in `times'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:173:in `block (2 levels) in <top (required)>'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/rake-10.5.0/lib/rake/task.rb:240:in `call'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/rake-10.5.0/lib/rake/task.rb:240:in `block in execute'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/rake-10.5.0/lib/rake/task.rb:235:in `each'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/rake-10.5.0/lib/rake/task.rb:235:in `execute'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/rake-10.5.0/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
  /Users/Chris/.rvm/rubies/ruby-2.2.1/lib/ruby/2.2.0/monitor.rb:211:in `mon_synchronize'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/rake-10.5.0/lib/rake/task.rb:172:in `invoke_with_call_chain'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/rake-10.5.0/lib/rake/task.rb:165:in `invoke'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/bin/derailed:41:in `exec'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/gems/derailed_benchmarks-1.3.1/bin/derailed:92:in `<top (required)>'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/bin/derailed:23:in `load'
  /Users/Chris/.rvm/gems/ruby-2.2.1@golf_mentor221/bin/derailed:23:in `<top (required)>'

My rails 4.2.6 project is installing derailed_benchmarks v0.0.0 when using rake >=11

I just created a new rails 4.2.6 project. After doing setup, I added:

group :development do
  gem 'stackprof'
  gem 'derailed'
end

And I tried to run it, but it fails:

/Users/MYUSER/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/bundler-1.11.2/lib/bundler/rubygems_integration.rb:355:in `block in replace_bin_path': can't find executable derailed (Gem::Exception)
    from /Users/MYUSER/.rbenv/versions/2.2.3/bin/derailed:23:in `<main>'

I noticed that in my Gemfile.lock says that derailed_benchmarks v0.0.0 is installed. I think that 0.0.0 is not a valid version for this gem.

To solve this, I downgraded to rake 10.5.0, and it seems to work.

I think there's some kind of issue with rake 11+

(in fact I noticed that in your gemspec the definition is: gem.add_dependency "rake", "~> 10")

Bad request when executing require_bench

Hi,

The require_bench rake task fails with a bad request error. What can I do to troubleshoot this ?

 bundle exec rake -f perf.rake perf:require_bench --trace
** Invoke perf:require_bench (first_time)
** Invoke perf:kernel_require_patch (first_time)
** Execute perf:kernel_require_patch
** Invoke perf:setup (first_time)
** Execute perf:setup
Booting: production
** Execute perf:require_bench
## Impact of `require <file>` on RAM

Showing all `require <file>` calls that consume 0.3 mb or more of RSS
Configure with `CUT_OFF=0` for all entries or `CUT_OFF=5` for few entries
Note: Files only count against RAM on their first load.
      If multiple libraries require the same file, then
       the 'cost' only shows up under the first library

rake aborted!
Bad request: 
/Users/ivan/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/derailed_benchmarks-0.0.0/lib/derailed_benchmarks/tasks.rb:56:in `call_app'
/Users/ivan/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/derailed_benchmarks-0.0.0/lib/derailed_benchmarks/tasks.rb:177:in `block (2 levels) in <top (required)>'
/Users/ivan/.rbenv/versions/2.0.0-p247/lib/ruby/gems/2.0.0/gems/rake-10.3.2/lib/rake/task.rb:240:in `call'

Memory allocated by gem

This is not an issue, one clarification I have been using this derailed gem to identify the memory issues and when i try this mem. Ruby 2.2.2, Rails 3.2.22

bundle exec derailed bundle:mem

TOP: 55.3398 MiB
delayed_job_active_record: 15.8789 MiB
delayed_job: 14.5352 MiB (Also required by: delayed/railtie)
delayed/yaml_ext: 10.4688 MiB

but when i try that objects

allocated memory by gem

35887974 activesupport-3.2.22
7594834 mime-types-1.25.1
7076454 newrelic_rpm-3.9.8.273
6541589 mail-2.5.4

I think there is more memory taking for this mime-types and newrelic, right? and if should i update the gem mail? i am following this link https://blog.codeship.com/debugging-a-memory-leak-on-heroku/

How to run derailed in a Rake Task or a Test

I'm trying to test cpu/memory/gc/etc consumption in some workers that I usually run in a rake task. I created a stress test with the code to test but I would like to use derailed_benchmark to get more info.

Is there anyway to do it?

Defining perf:rack_load task for a non-Rails app

I am trying to use derailed_benchmarks to dynamically benchmark my Padrino app. Following the docs I have created an appropriate perf:rack_load Rake task (code below).
However, when running bundle exec derailed exec perf:mem I get the following error:
/Users/me/.rvm/gems/ruby-2.3.1/gems/derailed_benchmarks-1.3.1/lib/derailed_benchmarks/tasks.rb:72:in block (2 levels) in <top (required)>': uninitialized constant DERAILED_APP (NameError)
It appears that when this task is invoked it simply runs the code from line 46 - and not my task.

Question: Is non-Rails/Rack actually supported right now - or now do I need to fork & overwrite the rack_load task at line 46? I see using config.ru was discussed in #1 - do you need a PR to make this so?

config.ru

#!/usr/bin/env rackup
# encoding: utf-8

require File.expand_path('../config/boot.rb', __FILE__)
use Rack::Deflater
run Padrino.application

perf.rake

require 'bundler'
Bundler.setup

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

namespace :perf do
  desc 'loads the app for derailed_benchmarks'
  task :rack_load do
    require_relative '../config/boot'
    DERAILED_APP = Padfoot::App
  end
end

`SystemStackError` when running `derailed exec perf:mem`

Hi, when I run derailed exec perf:mem I get the following error, what can I do?

Booting: production                                                                                                                                                                [9603/28895]
Traceback (most recent call last):                                                                                                                                                             
        9638: from /home/user/.rvm/gems/ruby-2.6.3@project/bin/ruby_executable_hooks:24:in `<main>'                                                                                         
        9637: from /home/user/.rvm/gems/ruby-2.6.3@project/bin/ruby_executable_hooks:24:in `eval'                                                                                           
        9636: from /home/user/.rvm/gems/ruby-2.6.3@project/bin/derailed:23:in `<main>'                                                                                                      
        9635: from /home/user/.rvm/gems/ruby-2.6.3@project/bin/derailed:23:in `load'                                                                                                        
        9634: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/bin/derailed:92:in `<top (required)>'                                                             
        9633: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/thor-0.20.3/lib/thor/base.rb:466:in `start'                                                                                 
        9632: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'                                                                                   
        9631: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/thor-0.20.3/lib/thor/invocation.rb:126:in `invoke_command'                                                                  
        9630: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/thor-0.20.3/lib/thor/command.rb:27:in `run'                                                                                 
        9629: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/bin/derailed:41:in `exec'                                                                         
        9628: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:183:in `invoke'                                                                  
        9627: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:194:in `invoke_with_call_chain'                                                  
        9626: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'                                                                                     
        9625: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:213:in `block in invoke_with_call_chain'                                         
        9624: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:236:in `invoke_prerequisites'                                                    
        9623: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:236:in `each'                                                                    
        9622: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:238:in `block in invoke_prerequisites'                                           
        9621: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:194:in `invoke_with_call_chain'                                                  
        9620: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'                                                                                     
        9619: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:214:in `block in invoke_with_call_chain'                                         
        9618: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:273:in `execute'                                                                 
        9617: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:273:in `each'
        9616: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:273:in `block in execute'
        9615: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/tasks.rb:60:in `block (2 levels) in <top (required)>'
        9614: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:183:in `invoke'
        9613: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:194:in `invoke_with_call_chain'
        9612: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
        9611: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:214:in `block in invoke_with_call_chain'
        9610: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:273:in `execute'
        9609: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:273:in `each'
        9608: from /home/user/.rvm/rubies/ruby-2.6.3/lib64/ruby/gems/2.6.0/gems/rake-12.3.2/lib/rake/task.rb:273:in `block in execute'
        9607: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/tasks.rb:19:in `block (2 levels) in <top (required)>'
        9606: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
        9605: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
        9604: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
        9603: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
        9602: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
        9601: from /home/user/src/govsciences/project/config/application.rb:3:in `<top (required)>'
        9600: from /home/user/src/govsciences/project/config/application.rb:3:in `require_relative'
        9599: from /home/user/src/govsciences/project/config/boot.rb:6:in `<top (required)>' 
        9598: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
        9597: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block in <top (required)>'
        9596: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:49:in `measure_memory_impact'
        9595: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `block (2 levels) in <top (required)>'
        9594: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:63:in `require'
        9593: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/setup.rb:27:in `<top (required)>'
        9592: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap.rb:24:in `setup'
        9591: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache.rb:47:in `setup'
        9590: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:48:in `require_relative'
        9589: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        9588: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require_with_bootsnap_lfi'
        9587: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
        9586: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `block in require_with_bootsnap_lfi'
        9585: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
        9584: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
        9583: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require_with_bootsnap_lfi'
        9582: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'

================== [omitted 1000s of repeated lines] ======================================

          11: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `block in require_with_bootsnap_lfi'
          10: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
           9: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
           8: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require_with_bootsnap_lfi'
           7: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
           6: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `block in require_with_bootsnap_lfi'
           5: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require'
           4: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
           3: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:21:in `require_with_bootsnap_lfi'
           2: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
           1: from /home/user/.rvm/gems/ruby-2.6.3@project/gems/bootsnap-1.4.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `block in require_with_bootsnap_lfi'
/home/user/.rvm/gems/ruby-2.6.3@project/gems/derailed_benchmarks-1.3.5/lib/derailed_benchmarks/core_ext/kernel_require.rb:16:in `require': stack level too deep (SystemStackError)

STDOUT log capture and emit errors on failure

Feature request

undefined method serialize_into_session

Since I am able to move forward by using the USE_SERVER=puma option, I am trying to benchmark an endpoint that requires authentication. This is an API that uses both Devise and token based authentication. So the requests to the API typically specify an authentication token.

I tried configuring perf.rake as described in the README:

require 'bundler'
Bundler.setup

require 'derailed_benchmarks'
require 'derailed_benchmarks/tasks'

DerailedBenchmarks.auth.user = -> { User.find_by(email: "[email protected]") }

But my request:

SE_AUTH=true PATH_TO_HIT=/api/v1/myendpoint RAILS_ENV=development USE_SERVER=puma rake -f perf.rake perf:mem

...fails with the following backtrace:

Started GET "/api/v1/myendpoint" for 127.0.0.1 at 2016-01-29 13:40:19 -0800

NoMethodError (undefined method `serialize_into_session' for #<Class:0x007fb21c1094f8>):
  devise (3.5.2) lib/devise.rb:448:in `block (2 levels) in configure_warden!'
  warden (1.2.3) lib/warden/session_serializer.rb:26:in `store'
  warden (1.2.3) lib/warden/proxy.rb:175:in `set_user'
  warden (1.2.3) lib/warden/test/helpers.rb:20:in `block in login_as'
  warden (1.2.3) lib/warden.rb:38:in `call'
  warden (1.2.3) lib/warden.rb:38:in `block in test_mode!'
  warden (1.2.3) lib/warden/hooks.rb:14:in `call'
  warden (1.2.3) lib/warden/hooks.rb:14:in `block in _run_callbacks'
  warden (1.2.3) lib/warden/hooks.rb:9:in `each'
  warden (1.2.3) lib/warden/hooks.rb:9:in `_run_callbacks'
  warden (1.2.3) lib/warden/manager.rb:53:in `_run_callbacks'
  warden (1.2.3) lib/warden/proxy.rb:31:in `initialize'
  warden (1.2.3) lib/warden/manager.rb:33:in `new'
  warden (1.2.3) lib/warden/manager.rb:33:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/etag.rb:24:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/conditionalget.rb:25:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/head.rb:13:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/params_parser.rb:27:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/flash.rb:260:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/session/abstract/id.rb:225:in `context'
  rack (1.6.4) lib/rack/session/abstract/id.rb:220:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/cookies.rb:560:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  activesupport (4.2.4) lib/active_support/callbacks.rb:88:in `__run_callbacks__'
  activesupport (4.2.4) lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
  activesupport (4.2.4) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (4.2.4) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/reloader.rb:73:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  web-console (2.2.1) lib/web_console/middleware.rb:39:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  railties (4.2.4) lib/rails/rack/logger.rb:38:in `call_app'
  railties (4.2.4) lib/rails/rack/logger.rb:20:in `block in call'
  activesupport (4.2.4) lib/active_support/tagged_logging.rb:68:in `block in tagged'
  activesupport (4.2.4) lib/active_support/tagged_logging.rb:26:in `tagged'
  activesupport (4.2.4) lib/active_support/tagged_logging.rb:68:in `tagged'
  railties (4.2.4) lib/rails/rack/logger.rb:20:in `call'
  quiet_assets (1.1.0) lib/quiet_assets.rb:27:in `call_with_quiet_assets'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/request_id.rb:21:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/methodoverride.rb:22:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/runtime.rb:18:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  activesupport (4.2.4) lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/lock.rb:17:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  actionpack (4.2.4) lib/action_dispatch/middleware/static.rb:116:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  rack (1.6.4) lib/rack/sendfile.rb:113:in `call'
  newrelic_rpm (3.13.2.302) lib/new_relic/agent/instrumentation/middleware_tracing.rb:67:in `call'
  railties (4.2.4) lib/rails/engine.rb:518:in `call'
  railties (4.2.4) lib/rails/application.rb:165:in `call'
  derailed_benchmarks (1.3.0) lib/derailed_benchmarks/auth_helpers/devise.rb:34:in `call'
  puma (2.14.0) lib/puma/server.rb:541:in `handle_request'
  puma (2.14.0) lib/puma/server.rb:388:in `process_client'
  puma (2.14.0) lib/puma/server.rb:270:in `block in run'
  puma (2.14.0) lib/puma/thread_pool.rb:106:in `call'
  puma (2.14.0) lib/puma/thread_pool.rb:106:in `block in spawn_thread'

So that got me wondering...does the USE_AUTH option work with USE_SERVER option?

Could you include version requirements?

I could create a pull request to get this working in 1.9.3 with Rails 3.2, but in the meantime, could you provide the versions that are known to work with this gem? I really look forward to using this if I can get it running in our environment. Thanks for the work so far.

Windows supported?

I have Windows 7 64bits.

When I run: "bundle exec derailed bundle:mem"
or "bundle exec derailed bundle:mem development"

I the following printed over and over again (I suspect once per gem loaded?) and that's it. This seems to indicate that ps command requested is not supported on Windows? A message in the readme would be nice if this is the case.

Usage ps [-aefl] [-u uid]
-f = show process uids, ppids
-l = show process uids, ppids, pgids, winpids
-u uid = list processes owned by uid
-a, -e = show processes of all users
-s = show process summary
-W = show windows as well as cygwin processes

PS: gem 'stackprof' doesn't even compile on Windows.

Error message is wonky when app returns non-200 status

There seems to be some weirdness going on if the app responds with a non-200 status.

1. There is a bundler issue reported as an error, even though obviously derailed has already been loaded and is halfway through being used.
2. The response.body output doesn't seem to be correct.

After running bundle exec derailed exec perf:ips

Warming up --------------------------------------
                 ips
bundler: failed to load command: derailed (/Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bin/derailed)
RuntimeError: Bad request: 
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bundler/gems/derailed_benchmarks-dcccae665e75/lib/derailed_benchmarks/tasks.rb:106:in `call_app'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bundler/gems/derailed_benchmarks-dcccae665e75/lib/derailed_benchmarks/tasks.rb:207:in `block (4 levels) in <top (required)>'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips/job/entry.rb:53:in `call_times'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips/job.rb:220:in `block in run_warmup'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips/job.rb:206:in `each'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips/job.rb:206:in `run_warmup'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips/job.rb:185:in `block in run'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips/job.rb:184:in `times'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips/job.rb:184:in `run'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/benchmark-ips-2.7.2/lib/benchmark/ips.rb:59:in `ips'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bundler/gems/derailed_benchmarks-dcccae665e75/lib/derailed_benchmarks/tasks.rb:206:in `block (2 levels) in <top (required)>'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/rake-12.3.0/lib/rake/task.rb:251:in `block in execute'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/rake-12.3.0/lib/rake/task.rb:251:in `each'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/rake-12.3.0/lib/rake/task.rb:251:in `execute'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/rake-12.3.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/rake-12.3.0/lib/rake/task.rb:188:in `invoke_with_call_chain'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/rake-12.3.0/lib/rake/task.rb:181:in `invoke'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bundler/gems/derailed_benchmarks-dcccae665e75/bin/derailed:41:in `exec'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/command.rb:27:in `run'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/invocation.rb:126:in `invoke_command'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor.rb:369:in `dispatch'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/gems/thor-0.19.4/lib/thor/base.rb:444:in `start'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bundler/gems/derailed_benchmarks-dcccae665e75/bin/derailed:92:in `<top (required)>'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bin/derailed:23:in `load'
  /Users/user/.rbenv/versions/2.4.3/lib/ruby/gems/2.4.0/bin/derailed:23:in `<top (required)>'

NameError: uninitialized constant ActiveRecord

Hey everyone, I tried to get this to run, specifically I tried to use the bundle exec rake -f perf.rake perf:require_bench command but when doing so I get the following:

rake aborted!
NameError: uninitialized constant ActiveRecord

Tasks: TOP => perf:require_bench => perf:setup
(See full trace by running task with --trace)

When ran with the trace I get:

bundle exec rake -f perf.rake perf:require_bench --trace
** Invoke perf:require_bench (first_time)
** Invoke perf:kernel_require_patch (first_time)
** Execute perf:kernel_require_patch
** Invoke perf:setup (first_time)
** Execute perf:setup
rake aborted!
NameError: uninitialized constant ActiveRecord
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/derailed_benchmarks-0.0.0/lib/derailed_benchmarks/tasks.rb:30:in `block (2 levels) in <top (required)>'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:240:in `call'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:240:in `block in execute'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:235:in `each'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:235:in `execute'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:172:in `invoke_with_call_chain'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:201:in `block in invoke_prerequisites'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:199:in `each'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:199:in `invoke_prerequisites'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:178:in `block in invoke_with_call_chain'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:172:in `invoke_with_call_chain'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/task.rb:165:in `invoke'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:150:in `invoke_task'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:106:in `each'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:106:in `block in top_level'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:115:in `run_with_threads'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:100:in `top_level'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:78:in `block in run'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:176:in `standard_exception_handling'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/lib/rake/application.rb:75:in `run'
/Users/KevinKirsch/.rvm/rubies/ruby-2.1.2/lib/ruby/gems/2.1.0/gems/rake-10.3.2/bin/rake:33:in `<top (required)>'
/Users/KevinKirsch/.rvm/gems/ruby-2.1.2/bin/rake:23:in `load'
/Users/KevinKirsch/.rvm/gems/ruby-2.1.2/bin/rake:23:in `<main>'
Tasks: TOP => perf:require_bench => perf:setup

Any suggestions on what I can do to get this to work or to disable it attempting to benchmark ActiveRecord which isn't in use in our application. Thank you for your time, I appreciate your patience.

Fails with 500 error for unknown reason

I'm getting started using this gem with my Rails app. I'm trying to profile the memory usage of specific endpoints. Using the default configuration (in development mode), I get an error:

$ RAILS_ENV=development rake -f perf.rake perf:mem

W, [2016-01-29T13:19:28.775029 #59970]  WARN -- : [SKYLIGHT] [0.10.0] Running Skylight in development mode. No data will be reported until you deploy your app.
(To disable this message for all local apps, run `skylight disable_dev_warning`.)
Endpoint: "/"
/Users/andrew/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.0/lib/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
/Users/andrew/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.0/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here
## Impact of `require <file>` on RAM

Showing all `require <file>` calls that consume 0.3 MiB or more of RSS
Configure with `CUT_OFF=0` for all entries or `CUT_OFF=5` for few entries
Note: Files only count against RAM on their first load.
      If multiple libraries require the same file, then
       the 'cost' only shows up under the first library

rake aborted!
Bad request: <!DOCTYPE html>
<html>
<head>
  <title>We're sorry, but something went wrong (500)</title>
  <meta name="viewport" content="width=device-width,initial-scale=1">
  <style>
  body {
    background-color: #EFEFEF;
    color: #2E2F30;
    text-align: center;
    font-family: arial, sans-serif;
    margin: 0;
  }

  div.dialog {
    width: 95%;
    max-width: 33em;
    margin: 4em auto 0;
  }

  div.dialog > div {
    border: 1px solid #CCC;
    border-right-color: #999;
    border-left-color: #999;
    border-bottom-color: #BBB;
    border-top: #B00100 solid 4px;
    border-top-left-radius: 9px;
    border-top-right-radius: 9px;
    background-color: white;
    padding: 7px 12% 0;
    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);
  }

  h1 {
    font-size: 100%;
    color: #730E15;
    line-height: 1.5em;
  }

  div.dialog > p {
    margin: 0 0 1em;
    padding: 1em;
    background-color: #F7F7F7;
    border: 1px solid #CCC;
    border-right-color: #999;
    border-left-color: #999;
    border-bottom-color: #999;
    border-bottom-left-radius: 4px;
    border-bottom-right-radius: 4px;
    border-top-color: #DADADA;
    color: #666;
    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);
  }
  </style>
</head>

<body>
  <!-- This file lives in public/500.html -->
  <div class="dialog">
    <div>
      <h1>We're sorry, but something went wrong.</h1>
    </div>
    <p>If you are the application owner check the logs for more information.</p>
  </div>
</body>
</html>

Tasks: TOP => perf:mem
(See full trace by running task with --trace)

The output in my development.log only shows the initial request:

Started GET "/" for  at 2016-01-29 13:20:49 -0800

However, I see that it works if I specify to use puma as the server:

$ RAILS_ENV=development USE_SERVER=puma rake -f perf.rake perf:mem
Booting: development
W, [2016-01-29T13:24:01.680161 #68693]  WARN -- : [SKYLIGHT] [0.10.0] Running Skylight in development mode. No data will be reported until you deploy your app.
(To disable this message for all local apps, run `skylight disable_dev_warning`.)
Endpoint: "/"
/Users/andrew/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.0/lib/derailed_benchmarks/tasks.rb:72: warning: already initialized constant DERAILED_APP
/Users/andrew/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/derailed_benchmarks-1.3.0/lib/derailed_benchmarks/tasks.rb:23: warning: previous definition of DERAILED_APP was here
Port: 3312
Server: "puma"
Puma 2.14.0 starting...
* Min threads: 0, max threads: 16
* Environment: none
* Listening on tcp://0.0.0.0:3312
## Impact of `require <file>` on RAM

Showing all `require <file>` calls that consume 0.3 MiB or more of RSS
Configure with `CUT_OFF=0` for all entries or `CUT_OFF=5` for few entries
Note: Files only count against RAM on their first load.
      If multiple libraries require the same file, then
       the 'cost' only shows up under the first library

TOP: 274.875 MiB
  application: 83.9492 MiB
    refile/rails: 18.0625 MiB
      ...etc...

difference between TOP vs application

hi,

running derailed in "local" production (RAILS_ENV=production bundle exec derailed exec perf:mem) gives me this :

TOP: 173.7539 MiB
application: 26.3008 MiB

while in dev I have (bundle exec derailed bundle:mem development) :

TOP: 48.4063 MiB
rails/all: 18.8047 MiB

My app do take about 180 mib on heroku. I am desperately trying to low this value. Any clue about where to look ? what does TOP mean ?

Thk U !!

perf:ram_over_time is stopping after couple of times

RAILS_ENV=production SKIP_FILE_WRITE=true rake -f perf.rake perf:ram_over_time TEST_COUNT=20
Doesn't matter what settings i changing it is silently stops.