从网上找到一处实例，跟着先把机器学习的数据分析的流程先了解一遍，把此实例的每一步理解记录一下。
数据集下载链接：https://github.com/susanli2016/Machine-Learning-with-Python/blob/master/diabetes.csv

一步步慢慢理解ML-sklearn

---

from warnings import simplefilter
simplefilter(action='ignore', category=FutureWarning)

上面这个代码块可以放在.py文件的开头，防止在运行过程中会出现不必要的警告。

---

import pandas as pd
diabetes = pd.read_csv("diabetes.csv")
print(diabetes.columns)  # check the column names

导入数据集，输出该数据集每列的类名，如下：

Index(['Pregnancies', 'Glucose', 'BloodPressure', 'SkinThickness', 'Insulin',
       'BMI', 'DiabetesPedigreeFunction', 'Age', 'Outcome'],
      dtype='object')

---

diabetes.head()     # default n = 5，show five rows
# the results 0 means not having diabetes,1 means having diabetes
# get the cows and cols of diabetes
print("dimension of diabetes data: {}".format(diabetes.shape)) 
# group the "Outcome" data to get the number of each group
print(diabetes.groupby('Outcome').size())  

diabetes.csv 文件中，"Outcome"这一列中0代表没有糖尿病，1代表有糖尿病。
代码输出俩个结果：csv文件的行数和列数；"Outcome"这一列中0，1的统计情况。

dimension of diabetes data: (768, 9)
Outcome
0    500
1    268
dtype: int64

---

import seaborn as sns
import matplotlib.pyplot as plt
sns.countplot(diabetes['Outcome'], label="Count")
plt.savefig("糖尿病数据处理图片/0_1_graph")

对"Outcome"这一列的数据进行绘图：

---

from sklearn.model_selection import train_test_split
diabetes.info()     # get the diatebes' info
x_train, x_test, y_train, y_test = train_test_split(
    diabetes.loc[:, diabetes.columns != 'Outcome'],
    diabetes['Outcome'], stratify=diabetes['Outcome'],
    random_state=66)

对数据集进行分隔，随机分为训练子集和测试子集。

---

KNN算法

from sklearn.neighbors import KNeighborsClassifier

training_accuracy = []
test_accuracy = []
# try n_neighbors from 1 to 10
neighbors_settings = range(1, 11)
for n_neighbors in neighbors_settings:
    knn = KNeighborsClassifier(n_neighbors=n_neighbors)  # build the models
    knn.fit(x_train, y_train)  # use x_train as train data and y_train as target value
    training_accuracy.append(knn.score(x_train, y_train))  # record training set accuracy
    test_accuracy.append(knn.score(x_test, y_test))  # record test set accuracy

'''
The relationship between the training set and the test set on the model prediction
accuracy (Y-axis) and the number of nearest neighbors (X-axis) is demonstrated
'''
plt.figure()
plt.plot(neighbors_settings, training_accuracy, label="training accuracy")
plt.plot(neighbors_settings, test_accuracy, label="test accuracy")
plt.ylabel("Accuracy")
plt.xlabel("n_neighbors")
plt.legend()
plt.savefig("糖尿病数据处理图片/knn_compare_model")

使用KNN算法寻找训练集和测试集在模型预测准确度（y轴）和近邻点个数（x轴）设置之间的关系。得到的关系图片如下：

为了使训练集和测试集的准确度都最高，我们选择9个邻近点，即n_neghbors=9。

---

# select n_neighbors = 9
knn = KNeighborsClassifier(n_neighbors=9)
knn.fit(x_train, y_train)
print("Accuracy of K-NN classifier on training set: {:.2f}".format(knn.score(x_train, y_train)))
print("Accuracy of K-NN classifier on test set: {:.2f}".format(knn.score(x_test, y_test)))

在上一步的基础上，我们选择了n_neighbors=9来进行KNN算法模拟，输出KNN分类训练的准确度：

Accuracy of K-NN classifier on training set: 0.79
Accuracy of K-NN classifier on test set: 0.78

---

线性逻辑回归

from sklearn.linear_model import LogisticRegression
# logistic regression analysis
"""
The accuracy of the model with regularization parameter C=1(default value) was 78%
the training set and 77% on the test set
"""
logreg = LogisticRegression(solver='liblinear').fit(x_train, y_train)
print("Training set score : {:.3f}".format(logreg.score(x_train, y_train)))
print("Test set score: {:.3f}".format(logreg.score(x_test, y_test)))

对数据再进行逻辑回归训练，得到的训练准确度如下：

Training set score : 0.781
Test set score: 0.771

在这里可以设置正则化参数C的值，默认值为1，当我们设置为C=100时，

"""
When the regularization parameter C is set to 100,
the accuracy of the model on the training set is slightly improved.
but the accuracy on the test set is slightly reduced.
"""
logreg100 = LogisticRegression(C=100, solver='liblinear').fit(x_train, y_train)
print("Training set score : {:.3f}".format(logreg100.score(x_train, y_train)))
print("Test set score: {:.3f}".format(logreg100.score(x_test, y_test)))

训练集的准确度有所提升但测试集的准确度下降，说明C=1比C=100好。

Training set score : 0.785
Test set score: 0.766

当C=0.001时，得到的准确度如下：

logreg001 = LogisticRegression(C=0.001, solver='liblinear').fit(x_train, y_train)
print("Training set score : {:.3f}".format(logreg001.score(x_train, y_train)))
print("Test set score: {:.3f}".format(logreg001.score(x_test, y_test)))

Training set score : 0.686
Test set score: 0.714

接下来用可视化的方法来看看不同正则化参数下所得的模型系数。

diabetes_features = [x for i, x in enumerate(diabetes.columns) if i != 8]

plt.figure(figsize=(8, 6))
plt.plot(logreg.coef_.T, 'o', label="C=1")
plt.plot(logreg100.coef_.T, 'd', label="C=100")
plt.plot(logreg001.coef_.T, '*', label="C=0.001")
plt.xticks(range(diabetes.shape[1]), diabetes_features, rotation=90)
plt.hlines(0, 0, diabetes.shape[1])
plt.ylim(-5, 5)
plt.xlabel("Feature")
plt.ylabel("Coefficient magnitude")
plt.legend()
plt.savefig("糖尿病数据处理图片/log_coef")

可以看到“DiabetesPedigreeFunction”（糖尿病遗传函数）在 C=100, C=1 和C=0.001的情况下, 系数都为正。这表明无论是哪个模型，DiabetesPedigreeFunction（糖尿病遗传函数）这个特征值都与样本为糖尿病是正相关的。

---

决策树

from sklearn.tree import DecisionTreeClassifier
# decision tree
"""
The results show that the accuracy of the training set is 100%,
while the accuracy of the test set is only 74.1%. So the decision tree is over-fitting.
"""
tree = DecisionTreeClassifier(random_state=0)
tree.fit(x_train, y_train)
print("\nPreliminary results of decision tree fitting:")
print("Accuracy on training set: {:.3f}".format(tree.score(x_train, y_train)))
print("Accuract on test set: {:.3f}".format(tree.score(x_test, y_test)))

对数据集进行决策树分析，发现训练集的准确度达到了100%，而测试集的准确度只有71.4%。所以我们要限制树的深度来减少过度拟合。

Preliminary results of decision tree fitting:
Accuracy on training set: 1.000
Accuract on test set: 0.714

设置max_depth=3，代码和输出结果如下：

# setting max_depth=3, limit the depth of the tree to reduce overfitting.
tree = DecisionTreeClassifier(max_depth=3, random_state=0)
tree.fit(x_train, y_train)
print("\nFinally results of decision tree fitting:")
print("Accuracy on training set: {:.3f}".format(tree.score(x_train, y_train)))
print("Accuract on test set: {:.3f}".format(tree.score(x_test, y_test)))
# output the feature importance
print("Feature importance:\n{}".format(tree.feature_importances_))

看到训练集准确度有所降低，但测试集准确度升高了。此外还输出了决策树中的特征重要性——决策树的特征重要性用来衡量每个特征对预测结果的重要性的。

Finally results of decision tree fitting:
Accuracy on training set: 0.773
Accuract on test set: 0.740
Feature importance:
[0.04554275 0.6830362  0.         0.         0.         0.27142106
 0.         0.        ]

对特征重要性进行数据可视化处理：

def plot_feature_importances_diatebes(model):
    plt.figure(figsize=(8, 6))
    n_features = 8
    plt.barh(range(n_features), model.feature_importances_, align='center')
    plt.yticks(np.arange(n_features), diabetes_features)
    plt.xlabel("Features importance")
    plt.ylabel("Feature")
    plt.ylim(-1, n_features)


plot_feature_importances_diatebes(tree)
plt.savefig("糖尿病数据处理图片/feature_importance")

直观的看出，Glucose（葡萄糖）是目前最重要的特征。BMI（身体质量指数）为第二重要的信息特征。

---

随机森林

from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
"""
A random forest of 100 trees was used to centralize the diabetes data
"""
# here I use default settings
rf = RandomForestClassifier(n_estimators=100, random_state=0)
rf.fit(x_train, y_train)
print("\nPreliminary results of decision tree fitting:")
print("Accuracy on training set: {:.3f}".format(rf.score(x_train, y_train)))
print("Accuracy on test set: {:.3f}".format(rf.score(x_test, y_test)))

使用100课树组成的随机森林对数据集进行分析。得到训练集的准确度为100%，测试集有78.6%，比逻辑回归和单一决策树都要好，但是我们还可以调节max_depth的值，看准确度是否能再提升。

Preliminary results of decision tree fitting:
Accuracy on training set: 1.000
Accuracy on test set: 0.786

# now, the max_depth = 3
rf1 = RandomForestClassifier(max_depth=3, n_estimators=100, random_state=0)
rf1.fit(x_train, y_train)
print("\nFinally results of decision tree fitting:")
print("Accuracy on training set: {:.3f}".format(rf1.score(x_train, y_train)))
print("Accuracy on test set: {:.3f}".format(rf1.score(x_test, y_test)))
"""By the results we could find the accuracy is reduced"""

Finally results of decision tree fitting:
Accuracy on training set: 0.800
Accuracy on test set: 0.755

可以看到，调整max_depth后，测试集的准确度下降了，因此，还是默认情况下准确度更好。
对随机森林进行特征重要度可视化：

plot_feature_importances_diatebes(rf)
plt.savefig("糖尿病数据处理图片/feature_importance_rf")

结果与单一决策树类似，随机森林的结果仍然显示特征"Glucose"（葡萄糖）的重要度最高。BMI（身体质量指数）为第二重要的信息特征。

---

梯度提升

gb = GradientBoostingClassifier(random_state=0)
gb.fit(x_train, y_train)
print("\nPreliminary results of decision tree fitting:")
print("Accuracy on training set: {:.3f}".format(gb.score(x_train, y_train)))
print("Accuracy on test set: {:.3f}".format(gb.score(x_test, y_test)))
plot_feature_importances_diatebes(gb)
plt.savefig("糖尿病数据处理图片/feature_importance_gb")

Preliminary results of decision tree fitting:
Accuracy on training set: 0.917
Accuracy on test set: 0.792

训练准确度太高，可能过拟合了。接下来通过限制最大深度和降低学习效率来进行修减。

限制最大深度

gb1 = GradientBoostingClassifier(max_depth=1, random_state=0)
gb1.fit(x_train, y_train)
print("\nFinally results of decision tree fitting:")
print("Accuracy on training set: {:.3f}".format(gb1.score(x_train, y_train)))
print("Accuracy on test set: {:.3f}".format(gb1.score(x_test, y_test)))
plot_feature_importances_diatebes(gb1)
plt.savefig("糖尿病数据处理图片/feature_importance_gb1")

Finally results of decision tree fitting:
Accuracy on training set: 0.804
Accuracy on test set: 0.781

降低学习效率

gb2 = GradientBoostingClassifier(random_state=0, learning_rate=0.01)
gb2.fit(x_train, y_train)
print("Accuracy on training set: {:.3f}".format(gb2.score(x_train, y_train)))
print("Accuracy on test set: {:.3f}".format(gb2.score(x_test, y_test)))
plot_feature_importances_diatebes(gb2)
plt.savefig("糖尿病数据处理图片/feature_importance_gb2")

Accuracy on training set: 0.802
Accuracy on test set: 0.776

可以观察到，两种方法都降低了训练集的准确度，但都没有提高测试集的准确度。
但是，我们可以根据特征重要度图来观察是否结果和前面的类似。可以发现，不管怎么进行修改，“Glucose”（葡萄糖）仍然是最重要的信息特征，"BMI"仍然是第二重要的信息特征。

---

向量机

---

x_train = x_train.astype(np.float64)
y_train = y_train.astype(np.float64)
x_test = x_test.astype(np.float64)
y_test = y_test.astype(np.float64)

先对数据进行类型处理，不然会在运行过程中显示警告，即使它不影响运行结果。

---

from sklearn.svm import SVC
svc = SVC()
svc.fit(x_train, y_train)
print("Accuracy on training set: {:.2f}".format(svc.score(x_train, y_train)))
print("Accuracy on test set: {:.2f}".format(svc.score(x_test, y_test)))

使用向量机对数据集进行拟合，得到准确度为：

Accuracy on training set: 1.00
Accuracy on test set: 0.65

这个结果显然过度拟合，虽然训练集的准确度有100%，但是测试集的准确度只有65%。因此，需要调整各特征值的尺度使其在同一量表上。

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_test_scaled = scaler.fit_transform(x_test)

然后再进行拟合：

svc = SVC()
svc.fit(x_train_scaled, y_train)
print("Accuracy on training set: {:.2f}".format(svc.score(x_train_scaled, y_train)))
print("Accuracy on test set: {:.2f}".format(svc.score(x_test_scaled, y_test)))

Accuracy on training set: 0.77
Accuracy on test set: 0.77

这是在参数默认的情况下的拟合结果，我们再试一试提高C值后的结果。令C=1000.

svc = SVC(C=1000)
svc.fit(x_train_scaled, y_train)
print("Accuracy on training set: {:.3f}".format(svc.score(x_train_scaled, y_train)))
print("Accuracy on test set: {:.3f}".format(svc.score(x_test_scaled, y_test)))

Accuracy on training set: 0.790
Accuracy on test set: 0.797

显然观察到，训练集和测试集的准确度都有所提高。

---

深度学习

from sklearn.neural_network import MLPClassifier
mlp = MLPClassifier(random_state=42)
mlp.fit(x_train, y_train)
print("Accuracy on training set: {:.2f}".format(mlp.score(x_train, y_train)))
print("Accuracy on test set: {:.2f}".format(mlp.score(x_test, y_test)))

Accuracy on training set: 0.73
Accuracy on test set: 0.72

多层神经网络（MLP）的表现并没有其他的模型好，所以要对数据进行处理。

# Standardized data
scaler = StandardScaler()
x_train_scaled = scaler.fit_transform(x_train)
x_test_scaled = scaler.fit_transform(x_test)

mlp = MLPClassifier(random_state=0)
mlp.fit(x_train_scaled, y_train)
print("Accuracy on training set: {:.3f}".format(mlp.score(x_train_scaled, y_train)))
print("Accuracy on test set: {:.3f}".format(mlp.score(x_test_scaled, y_test)))

Accuracy on training set: 0.823
Accuracy on test set: 0.802

增加迭代次数：

mlp = MLPClassifier(max_iter=10000, random_state=0)
mlp.fit(x_train_scaled, y_train)
print("Accuracy on training set: {:.3f}".format(mlp.score(x_train_scaled, y_train)))
print("Accuracy on test set: {:.3f}".format(mlp.score(x_test_scaled, y_test)))

Accuracy on training set: 1.000
Accuracy on test set: 0.771

发现增加迭代次数知识增加了训练集的准确度，而测试集的准确度还降低了。

调高alpha参数并且加强权重的正则化

mlp = MLPClassifier(max_iter=10000, alpha=1, random_state=0)
mlp.fit(x_train_scaled, y_train)
print("Accuracy on training set: {:.3f}".format(mlp.score(x_train_scaled, y_train)))
print("Accuracy on test set: {:.3f}".format(mlp.score(x_test_scaled, y_test)))

Accuracy on training set: 0.806
Accuracy on test set: 0.797

结果比上一个好，但没有第一次结果好。

最后，绘制了一张在糖尿病数据集上学习的神经网络的第一层权重热图。

plt.figure(figsize=(20, 5))
plt.imshow(mlp.coefs_[0], interpolation='none', cmap='viridis')
plt.yticks(range(8), diabetes_features)
plt.xlabel("Columns in weight matrix")
plt.ylabel("Input feature")
plt.colorbar()
plt.savefig("糖尿病数据处理图片/feature_hot")

热图很模糊，想要看清楚哪一个特征度的重要性是不容易的。