moelf / evotrees.jl

Boosted trees in Julia

EvoTrees

Documentation	CI Status

A Julia implementation of boosted trees with CPU and GPU support. Efficient histogram based algorithms with support for multiple loss functions (notably multi-target objectives such as max likelihood methods).

R binding available.

Input features are expected to be Matrix{Float64/Float32} when using the internal API. Tables/DataFrames format can be handled through MLJ. See the docs for further details.

Installation

Latest:

julia> Pkg.add("https://github.com/Evovest/EvoTrees.jl")

From General Registry:

julia> Pkg.add("EvoTrees")

Performance

Data consists of randomly generated float32. Training is performed on 200 iterations. Code to reproduce is here.

EvoTrees: v0.8.4 XGBoost: v1.1.1

CPU: 16 threads on AMD Threadripper 3970X GPU: NVIDIA RTX 2080

Training:

Dimensions / Algo	XGBoost Hist	EvoTrees	EvoTrees GPU
100K x 100	1.10s	1.80s	3.14s
500K x 100	4.83s	4.98s	4.98s
1M x 100	9.84s	9.89s	7.37s
5M x 100	45.5s	53.8s	25.8s

Inference:

Dimensions / Algo	XGBoost Hist	EvoTrees	EvoTrees GPU
100K x 100	0.164s	0.026s	0.013s
500K x 100	0.796s	0.175s	0.055s
1M x 100	1.59s	0.396s	0.108s
5M x 100	7.96s	2.15s	0.543s

MLJ Integration

See official project page for more info.

Getting started using internal API

using EvoTrees

config = EvoTreeRegressor(
    loss=:linear, 
    nrounds=100, 
    nbins=100,
    lambda=0.5, 
    gamma=0.1, 
    eta=0.1,
    max_depth=6, 
    min_weight=1.0,
    rowsample=0.5, 
    colsample=1.0)
model = fit_evotree(config; x_train = x_train, y_train = y_train)
preds = predict(model, x_eval)

Feature importance

Returns the normalized gain by feature.

features_gain = importance(model)

Plot

Plot a given tree of the model:

plot(model, 2)

Note that 1st tree is used to set the bias so the first real tree is #2.

Save/Load

EvoTrees.save(model, "data/model.bson")
model = EvoTrees.load("data/model.bson");