Regression with CART Trees - Lab
Introduction
In this lab, we'll make use of what we learned in the previous lesson to build a model for the Petrol Consumption Dataset from Kaggle. This model will be used to predict gasoline consumption for a bunch of examples, based on features about the drivers.
Objectives
In this lab you will:
- Fit a decision tree regression model with scikit-learn
Import necessary libraries
# Import libraries
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
The dataset
- Import the
'petrol_consumption.csv'
dataset - Print the first five rows of the data
- Print the dimensions of the data
# Import the dataset
dataset = None
# Print the first five rows
# Print the dimensions of the data
- Print the summary statistics of all columns in the data:
# Describe the dataset
Create training and test sets
- Assign the target column
'Petrol_Consumption'
toy
- Assign the remaining independent variables to
X
- Split the data into training and test sets using a 80/20 split
- Set the random state to 42
# Split the data into training and test sets
X = None
y = None
X_train, X_test, y_train, y_test = None
Create an instance of CART regressor and fit the data to the model
As mentioned earlier, for a regression task we'll use a different sklearn
class than we did for the classification task. The class we'll be using here is the DecisionTreeRegressor
class, as opposed to the DecisionTreeClassifier
from before.
# Import the DecisionTreeRegressor class
# Instantiate and fit a regression tree model to training data
regressor = None
Make predictions and calculate the MAE, MSE, and RMSE
Use the above model to generate predictions on the test set.
Just as with decision trees for classification, there are several commonly used metrics for evaluating the performance of our model. The most common metrics are:
- Mean Absolute Error (MAE)
- Mean Squared Error (MSE)
- Root Mean Squared Error (RMSE)
If these look familiar, it's likely because you have already seen them before -- they are common evaluation metrics for any sort of regression model, and as we can see, regressions performed with decision tree models are no exception!
Since these are common evaluation metrics, sklearn
has functions for each of them that we can use to make our job easier. You'll find these functions inside the metrics
module. In the cell below, calculate each of the three evaluation metrics.
from sklearn.metrics import mean_absolute_error, mean_squared_error
# Make predictions on the test set
y_pred = None
# Evaluate these predictions
print('Mean Absolute Error:', None)
print('Mean Squared Error:', None)
print('Root Mean Squared Error:', None)
Level Up (Optional)
-
Look at the hyperparameters used in the regression tree, check their value ranges in official doc and try running some optimization by growing a number of trees in a loop
-
Use a dataset that you are familiar with and run tree regression to see if you can interpret the results
-
Check for outliers, try normalization and see the impact on the output
Summary
In this lesson, you implemented the architecture to train a tree regressor and predict values for unseen data. You saw that with a vanilla approach, the results were not so great, and thus we must further tune the model (what we described as hyperparameter optimization and pruning, in the case of trees).