learn-co-students / dsc-regression-cart-trees-lab-wdc01-dtsc-ft-060120

Regression with CART Trees - Lab

Introduction

In this lab, we'll make use of what we learned in the previous lesson to build a model for the Petrol Consumption Dataset from Kaggle. This model will be used to predict gasoline consumption for a bunch of examples, based on features about the drivers.

Objectives

In this lab you will:

• Fit a decision tree regression model with scikit-learn

Import necessary libraries

# Import libraries 
import pandas as pd  
import numpy as np  
from sklearn.model_selection import train_test_split 

The dataset

• Import the 'petrol_consumption.csv' dataset
• Print the first five rows of the data
• Print the dimensions of the data

# Import the dataset
dataset = None

# Print the first five rows

# Print the dimensions of the data

• Print the summary statistics of all columns in the data:

# Describe the dataset

Create training and test sets

• Assign the target column 'Petrol_Consumption' to y
• Assign the remaining independent variables to X
• Split the data into training and test sets using a 80/20 split
• Set the random state to 42

# Split the data into training and test sets
X = None
y = None
X_train, X_test, y_train, y_test = None

Create an instance of CART regressor and fit the data to the model

As mentioned earlier, for a regression task we'll use a different sklearn class than we did for the classification task. The class we'll be using here is the DecisionTreeRegressor class, as opposed to the DecisionTreeClassifier from before.

# Import the DecisionTreeRegressor class 


# Instantiate and fit a regression tree model to training data 
regressor = None

Make predictions and calculate the MAE, MSE, and RMSE

Use the above model to generate predictions on the test set.

Just as with decision trees for classification, there are several commonly used metrics for evaluating the performance of our model. The most common metrics are:

• Mean Absolute Error (MAE)
• Mean Squared Error (MSE)
• Root Mean Squared Error (RMSE)

If these look familiar, it's likely because you have already seen them before -- they are common evaluation metrics for any sort of regression model, and as we can see, regressions performed with decision tree models are no exception!

Since these are common evaluation metrics, sklearn has functions for each of them that we can use to make our job easier. You'll find these functions inside the metrics module. In the cell below, calculate each of the three evaluation metrics.

from sklearn.metrics import mean_absolute_error, mean_squared_error

# Make predictions on the test set
y_pred = None

# Evaluate these predictions
print('Mean Absolute Error:', None)  
print('Mean Squared Error:', None)  
print('Root Mean Squared Error:', None)

Level Up (Optional)

• Look at the hyperparameters used in the regression tree, check their value ranges in official doc and try running some optimization by growing a number of trees in a loop

• Use a dataset that you are familiar with and run tree regression to see if you can interpret the results

• Check for outliers, try normalization and see the impact on the output

Summary

In this lesson, you implemented the architecture to train a tree regressor and predict values for unseen data. You saw that with a vanilla approach, the results were not so great, and thus we must further tune the model (what we described as hyperparameter optimization and pruning, in the case of trees).