breather / brightbox

Peek into any blackbox learner (including ensembles) with brightbox

brightbox

Documentation

https://github.com/breather/brightbox/blob/master/inst/doc/brightbox.pdf

Installation

require(devtools) 
install_github("breather/brightbox")

About

Brightbox is an R package that tackles the problem of determining variable importance in the context of supervised machine learning. Traditional variable importance methods often rely on theoretical assumptions that fail to hold in the real world.

Brightbox is guided by a "learner agnostic" philosophy:

1. The end user can specify any caret method as the supervised learning algorithm.
2. All measures of importance are based on looking at empirical changes when a variable is removed or changed in some way.

Sources

Paper describing motivation for calculate_marginal_vimp: https://arxiv.org/pdf/1701.04944v4.pdf

brightbox's Issues

Change axis labels

Change y-axis label to "partial dependence"
Change x-axis label to "value cutpoint"

explicitly map colors in facet plotting

just as line_type and size_type are mapped, create explicit aesthetics for colour of lines, e.g. ensemble is always black, etc

Create abstraction for plotting model quantiles

Create functions for creating quantiles of model predictions when dealing with too many models for sensible plotting

partial dependency plots break when special symbols are in column names

For example feature_col = "bucket_-6" results in the error object bucket_ not found

create better aesthetic for ensemble model

ensemble line should be better differentiated in partial dependency plot from component models

request for facet plot for calculate_pd_vimp

would be amazing to have a function that takes output of calculate_pd_vimp and creates a facet plot of partial dependency for top N most important features

allow_parallel = TRUE fails on calculate_pd_vimp

parallel fails locally on Mac OS X, brightbox version 0.0.0.9 with following message:

Error in data.table(var = var, vimp = vimp_range[2] - vimp_range[1]) : 
  column or argument 1 is NULL

mesh grid

would be ideal to have a two dimensional input, return heatmap or other 3d continuous visualization of partial dependency

remove pd_list and return long-format tables instead