chenwenxiang1019 / sklearn-decisiontree

决策树、随机森林

sklearn-DecisionTree

决策树、随机森林

用泰坦尼克号事件的数据集练习一下决策树和随机森林的API。

分析数据集信息

先读入数据集，看看有哪些特征：

import pandas as pd

titanic = pd.read_csv("titanic.csv")

# 分析数据集信息
print("*" * 30 + " info " + "*" * 30)
print(titanic.info())
print(titanic.head())

输出：

****************************** info ******************************
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 12 columns):
PassengerId    891 non-null int64
Survived       891 non-null int64
Pclass         891 non-null int64
Name           891 non-null object
Sex            891 non-null object
Age            714 non-null float64
SibSp          891 non-null int64
Parch          891 non-null int64
Ticket         891 non-null object
Fare           891 non-null float64
Cabin          204 non-null object
Embarked       889 non-null object
dtypes: float64(2), int64(5), object(5)
memory usage: 83.6+ KB
None
   PassengerId  Survived  Pclass    ...        Fare Cabin  Embarked
0            1         0       3    ...      7.2500   NaN         S
1            2         1       1    ...     71.2833   C85         C
2            3         1       3    ...      7.9250   NaN         S
3            4         1       1    ...     53.1000  C123         S
4            5         0       3    ...      8.0500   NaN         S

[5 rows x 12 columns]

survived字段表示是否生存，我们以此作为预测目标。

y = titanic['survived']
print(y.head())

输出：

0    0
1    1
2    1
3    1
4    0
Name: survived, dtype: int64

我们取其中三个特征做分析演示，分别是：

• pclass：1-一等舱，2-二等舱，3-三等舱
• age年龄
• sex性别

x = titanic[['pclass', 'age', 'sex']]
print(x.info())
print(x.head())

输出：

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 3 columns):
Pclass    891 non-null int64
Age       714 non-null float64
Sex       891 non-null object
dtypes: float64(1), int64(1), object(1)
memory usage: 21.0+ KB
None
   Pclass   Age     Sex
0       3  22.0    male
1       1  38.0  female
2       3  26.0  female
3       1  35.0  female
4       3  35.0    male

缺失值处理

age字段存在缺失，用均值填充：

age_mean = x['age'].mean()
print("*" * 30 + " age_mean " + "*" * 30)
print(age_mean)
x['age'].fillna(age_mean, inplace=True)
print("*" * 30 + " 处理age缺失值后 " + "*" * 30)
print(x.info())

输出：

****************************** age_mean ******************************
29.69911764705882
****************************** 处理age缺失值后 ******************************
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 3 columns):
Pclass    891 non-null int64
Age       891 non-null float64
Sex       891 non-null object
dtypes: float64(1), int64(1), object(1)
memory usage: 21.0+ KB

特征抽取 - onehot编码

为了方便使用字典特征抽取，构造字典列表：

x_dict_list = x.to_dict(orient='records')
print("*" * 30 + " train_dict " + "*" * 30)
print(pd.Series(x_dict_list[:5]))

dict_vec = DictVectorizer(sparse=False)
x = dict_vec.fit_transform(x_dict_list)
print("*" * 30 + " onehot编码 " + "*" * 30)
print(dict_vec.get_feature_names())
print(x[:5])

****************************** train_dict ******************************
0      {'Pclass': 3, 'Age': 22.0, 'Sex': 'male'}
1    {'Pclass': 1, 'Age': 38.0, 'Sex': 'female'}
2    {'Pclass': 3, 'Age': 26.0, 'Sex': 'female'}
3    {'Pclass': 1, 'Age': 35.0, 'Sex': 'female'}
4      {'Pclass': 3, 'Age': 35.0, 'Sex': 'male'}
dtype: object
****************************** onehot编码 ******************************
['Age', 'Pclass', 'Sex=female', 'Sex=male']
[[22.  3.  0.  1.]
 [38.  1.  1.  0.]
 [26.  3.  1.  0.]
 [35.  1.  1.  0.]
 [35.  3.  0.  1.]]

划分训练集和测试集

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.25)

决策树分类器

dec_tree = DecisionTreeClassifier()
dec_tree.fit(x_train, y_train)

print("*" * 30 + " 准确率 " + "*" * 30)
print(dec_tree.score(x_test, y_test))

输出：

****************************** 准确率 ******************************
0.7892376681614349

随机森林分类器

• n_jobs: -1表示设置为核心数量
• n_estimators: 决策树数目
• max_depth: 树最大深度

同时使用网格搜索最优超参数：

rf = RandomForestClassifier(n_jobs=-1)
param = {
    "n_estimators": [120, 200, 300, 500, 800, 1200],
    "max_depth": [5, 8, 15, 25, 30]
}
# 2折交叉验证
search = GridSearchCV(rf, param_grid=param, cv=2)
print("*" * 30 + " 超参数网格搜索 " + "*" * 30)

start_time = time.time()
search.fit(x_train, y_train)
print("耗时：{}".format(time.time() - start_time))
print("最优参数：{}".format(search.best_params_))

print("*" * 30 + " 准确率 " + "*" * 30)
print(search.score(x_test, y_test))

输出：

****************************** 超参数网格搜索 ******************************
耗时：66.85670185089111
最优参数：{'max_depth': 5, 'n_estimators': 120}
****************************** 准确率 ******************************
0.7847533632286996

最优的参数是{'max_depth': 5, 'n_estimators': 120}。 在我的2015款MacBookPro上，仅2折的交叉验证就跑了66秒。- -|