第三章模型建立和评估---评价-课程.ipynb

{"cells":[{"cell_type":"markdown","metadata":{"id":"MwWqDjwVcQfX"},"source":["## 第三章 模型搭建和评估-评估"]},{"cell_type":"markdown","metadata":{"id":"jUlAaLtmcQfb"},"source":["根据之前的模型的建模，我们知道如何运用sklearn这个库来完成建模，以及我们知道了的数据集的划分等等操作。那么一个模型我们怎么知道它好不好用呢？以至于我们能不能放心的使用模型给我的结果呢？那么今天的学习的评估，就会很有帮助。"]},{"cell_type":"markdown","metadata":{"id":"otnD_DZwcQfc"},"source":["加载下面的库"]},{"cell_type":"code","execution_count":1,"metadata":{"id":"zHsb6UVWcQfc","executionInfo":{"status":"ok","timestamp":1680522052991,"user_tz":-540,"elapsed":2240,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["import pandas as pd\n","import numpy as np\n","import seaborn as sns\n","import matplotlib.pyplot as plt\n","from IPython.display import Image\n","from sklearn.linear_model import LogisticRegression\n","from sklearn.ensemble import RandomForestClassifier"]},{"cell_type":"code","execution_count":2,"metadata":{"id":"BzZt_UVzcQfe","executionInfo":{"status":"ok","timestamp":1680522055755,"user_tz":-540,"elapsed":262,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["%matplotlib inline"]},{"cell_type":"code","execution_count":3,"metadata":{"id":"Ml_ZSTFhcQff","executionInfo":{"status":"ok","timestamp":1680522058140,"user_tz":-540,"elapsed":244,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["plt.rcParams['font.sans-serif'] = ['SimHei']  # 用来正常显示中文标签\n","plt.rcParams['axes.unicode_minus'] = False  # 用来正常显示负号\n","plt.rcParams['figure.figsize'] = (10, 6)  # 设置输出图片大小"]},{"cell_type":"markdown","metadata":{"id":"CP45MukOcQfg"},"source":["**任务：加载数据并分割测试集和训练集**"]},{"cell_type":"code","execution_count":4,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":270},"id":"btGX_xy0cQfg","executionInfo":{"status":"ok","timestamp":1680522061815,"user_tz":-540,"elapsed":247,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"2bd9708d-fa19-4210-b916-65d0bf53ad95"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["   PassengerId  Pclass   Age  SibSp  Parch     Fare  Sex_female  Sex_male  \\\n","0            0       3  22.0      1      0   7.2500           0         1   \n","1            1       1  38.0      1      0  71.2833           1         0   \n","2            2       3  26.0      0      0   7.9250           1         0   \n","3            3       1  35.0      1      0  53.1000           1         0   \n","4            4       3  35.0      0      0   8.0500           0         1   \n","\n","   Embarked_C  Embarked_Q  Embarked_S  \n","0           0           0           1  \n","1           1           0           0  \n","2           0           0           1  \n","3           0           0           1  \n","4           0           0           1  "],"text/html":["\n","  <div id=\"df-adac46a7-bcaa-4de8-8973-59dfcb6bb5ac\">\n","    <div class=\"colab-df-container\">\n","      <div>\n","<style scoped>\n","    .dataframe tbody tr th:only-of-type {\n","        vertical-align: middle;\n","    }\n","\n","    .dataframe tbody tr th {\n","        vertical-align: top;\n","    }\n","\n","    .dataframe thead th {\n","        text-align: right;\n","    }\n","</style>\n","<table border=\"1\" class=\"dataframe\">\n","  <thead>\n","    <tr style=\"text-align: right;\">\n","      <th></th>\n","      <th>PassengerId</th>\n","      <th>Pclass</th>\n","      <th>Age</th>\n","      <th>SibSp</th>\n","      <th>Parch</th>\n","      <th>Fare</th>\n","      <th>Sex_female</th>\n","      <th>Sex_male</th>\n","      <th>Embarked_C</th>\n","      <th>Embarked_Q</th>\n","      <th>Embarked_S</th>\n","    </tr>\n","  </thead>\n","  <tbody>\n","    <tr>\n","      <th>0</th>\n","      <td>0</td>\n","      <td>3</td>\n","      <td>22.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>7.2500</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","    </tr>\n","    <tr>\n","      <th>1</th>\n","      <td>1</td>\n","      <td>1</td>\n","      <td>38.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>71.2833</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","    </tr>\n","    <tr>\n","      <th>2</th>\n","      <td>2</td>\n","      <td>3</td>\n","      <td>26.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>7.9250</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","    </tr>\n","    <tr>\n","      <th>3</th>\n","      <td>3</td>\n","      <td>1</td>\n","      <td>35.0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>53.1000</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","    </tr>\n","    <tr>\n","      <th>4</th>\n","      <td>4</td>\n","      <td>3</td>\n","      <td>35.0</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>8.0500</td>\n","      <td>0</td>\n","      <td>1</td>\n","      <td>0</td>\n","      <td>0</td>\n","      <td>1</td>\n","    </tr>\n","  </tbody>\n","</table>\n","</div>\n","      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-adac46a7-bcaa-4de8-8973-59dfcb6bb5ac')\"\n","              title=\"Convert this dataframe to an interactive table.\"\n","              style=\"display:none;\">\n","        \n","  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n","       width=\"24px\">\n","    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n","    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n","  </svg>\n","      </button>\n","      \n","  <style>\n","    .colab-df-container {\n","      display:flex;\n","      flex-wrap:wrap;\n","      gap: 12px;\n","    }\n","\n","    .colab-df-convert {\n","      background-color: #E8F0FE;\n","      border: none;\n","      border-radius: 50%;\n","      cursor: pointer;\n","      display: none;\n","      fill: #1967D2;\n","      height: 32px;\n","      padding: 0 0 0 0;\n","      width: 32px;\n","    }\n","\n","    .colab-df-convert:hover {\n","      background-color: #E2EBFA;\n","      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n","      fill: #174EA6;\n","    }\n","\n","    [theme=dark] .colab-df-convert {\n","      background-color: #3B4455;\n","      fill: #D2E3FC;\n","    }\n","\n","    [theme=dark] .colab-df-convert:hover {\n","      background-color: #434B5C;\n","      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n","      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n","      fill: #FFFFFF;\n","    }\n","  </style>\n","\n","      <script>\n","        const buttonEl =\n","          document.querySelector('#df-adac46a7-bcaa-4de8-8973-59dfcb6bb5ac button.colab-df-convert');\n","        buttonEl.style.display =\n","          google.colab.kernel.accessAllowed ? 'block' : 'none';\n","\n","        async function convertToInteractive(key) {\n","          const element = document.querySelector('#df-adac46a7-bcaa-4de8-8973-59dfcb6bb5ac');\n","          const dataTable =\n","            await google.colab.kernel.invokeFunction('convertToInteractive',\n","                                                     [key], {});\n","          if (!dataTable) return;\n","\n","          const docLinkHtml = 'Like what you see? Visit the ' +\n","            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n","            + ' to learn more about interactive tables.';\n","          element.innerHTML = '';\n","          dataTable['output_type'] = 'display_data';\n","          await google.colab.output.renderOutput(dataTable, element);\n","          const docLink = document.createElement('div');\n","          docLink.innerHTML = docLinkHtml;\n","          element.appendChild(docLink);\n","        }\n","      </script>\n","    </div>\n","  </div>\n","  "]},"metadata":{},"execution_count":4}],"source":["#写入代码\n","X = pd.read_csv(\"/content/clear_data.csv\")\n","X.head()"]},{"cell_type":"code","execution_count":5,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"jMiYe-iAcQfg","executionInfo":{"status":"ok","timestamp":1680522068637,"user_tz":-540,"elapsed":442,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"c73cac9a-ad63-4d69-d95d-671e11c26b9c"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["0    0\n","1    1\n","2    1\n","3    1\n","4    0\n","Name: Survived, dtype: int64"]},"metadata":{},"execution_count":5}],"source":["#写入代码\n","y = pd.read_csv(\"/content/train.csv\")[\"Survived\"]\n","y.head()"]},{"cell_type":"code","execution_count":6,"metadata":{"id":"BSQ-6q6_cQfh","executionInfo":{"status":"ok","timestamp":1680522265393,"user_tz":-540,"elapsed":739,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["#写入代码\n","from sklearn.model_selection import train_test_split\n","X_train, X_test, y_train, y_test = train_test_split(X, y, stratify = y,random_state = 0)"]},{"cell_type":"code","execution_count":7,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"aQ7Ek5bfcQfh","executionInfo":{"status":"ok","timestamp":1680522267094,"user_tz":-540,"elapsed":287,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"34e2b2eb-4179-4934-997b-6274e70cc18a"},"outputs":[{"output_type":"stream","name":"stdout","text":["<class 'pandas.core.frame.DataFrame'>\n","Int64Index: 668 entries, 671 to 80\n","Data columns (total 11 columns):\n"," #   Column       Non-Null Count  Dtype  \n","---  ------       --------------  -----  \n"," 0   PassengerId  668 non-null    int64  \n"," 1   Pclass       668 non-null    int64  \n"," 2   Age          668 non-null    float64\n"," 3   SibSp        668 non-null    int64  \n"," 4   Parch        668 non-null    int64  \n"," 5   Fare         668 non-null    float64\n"," 6   Sex_female   668 non-null    int64  \n"," 7   Sex_male     668 non-null    int64  \n"," 8   Embarked_C   668 non-null    int64  \n"," 9   Embarked_Q   668 non-null    int64  \n"," 10  Embarked_S   668 non-null    int64  \n","dtypes: float64(2), int64(9)\n","memory usage: 62.6 KB\n"]},{"output_type":"execute_result","data":{"text/plain":["(None,\n"," 671    0\n"," 417    1\n"," 634    0\n"," 323    1\n"," 379    0\n","       ..\n"," 131    0\n"," 490    0\n"," 528    0\n"," 48     0\n"," 80     0\n"," Name: Survived, Length: 668, dtype: int64)"]},"metadata":{},"execution_count":7}],"source":["#写入代码\n","X_train.info(),y_train"]},{"cell_type":"code","source":["X_train.dtypes,y_train.dtypes"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"6J6OXTxUgN3v","executionInfo":{"status":"ok","timestamp":1680522272325,"user_tz":-540,"elapsed":265,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"88958e17-c3f1-4ac2-cbe7-0dd8c4d826ec"},"execution_count":8,"outputs":[{"output_type":"execute_result","data":{"text/plain":["(PassengerId      int64\n"," Pclass           int64\n"," Age            float64\n"," SibSp            int64\n"," Parch            int64\n"," Fare           float64\n"," Sex_female       int64\n"," Sex_male         int64\n"," Embarked_C       int64\n"," Embarked_Q       int64\n"," Embarked_S       int64\n"," dtype: object,\n"," dtype('int64'))"]},"metadata":{},"execution_count":8}]},{"cell_type":"code","source":["lr = LogisticRegression()\n","lr.fit(X_train, y_train)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":238},"id":"XJOx3hPogY48","executionInfo":{"status":"ok","timestamp":1680522276265,"user_tz":-540,"elapsed":250,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"0ccb6efe-49a0-4928-bcda-1220a2718cbb"},"execution_count":9,"outputs":[{"output_type":"stream","name":"stderr","text":["/usr/local/lib/python3.9/dist-packages/sklearn/linear_model/_logistic.py:458: ConvergenceWarning: lbfgs failed to converge (status=1):\n","STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n","\n","Increase the number of iterations (max_iter) or scale the data as shown in:\n","    https://scikit-learn.org/stable/modules/preprocessing.html\n","Please also refer to the documentation for alternative solver options:\n","    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n","  n_iter_i = _check_optimize_result(\n"]},{"output_type":"execute_result","data":{"text/plain":["LogisticRegression()"],"text/html":["<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression()</pre></div></div></div></div></div>"]},"metadata":{},"execution_count":9}]},{"cell_type":"code","source":["rfc = RandomForestClassifier(n_estimators=100, max_depth=5)\n","rfc.fit(X_train, y_train)"],"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":75},"id":"LMogoEltgjHF","executionInfo":{"status":"ok","timestamp":1680522280091,"user_tz":-540,"elapsed":1010,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"0a020cc8-085f-4b0f-ca7d-080103011dd6"},"execution_count":10,"outputs":[{"output_type":"execute_result","data":{"text/plain":["RandomForestClassifier(max_depth=5)"],"text/html":["<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestClassifier(max_depth=5)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(max_depth=5)</pre></div></div></div></div></div>"]},"metadata":{},"execution_count":10}]},{"cell_type":"code","source":["print(\"Training set score:{:.2f}\".format(rfc.score(X_train, y_train)))\n","print(\"Training set score:{:.2f}\".format(rfc.score(X_test, y_test)))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"p7v8mieag7hv","executionInfo":{"status":"ok","timestamp":1680522282318,"user_tz":-540,"elapsed":248,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"59f4013a-9bc5-4301-b317-53e1138a61d7"},"execution_count":11,"outputs":[{"output_type":"stream","name":"stdout","text":["Training set score:0.87\n","Training set score:0.82\n"]}]},{"cell_type":"markdown","metadata":{"id":"HuhyiYXUcQfh"},"source":["### 模型评估"]},{"cell_type":"markdown","metadata":{"id":"zYM3q6YlcQfi"},"source":["* 模型评估是为了知道模型的泛化能力。\n","* 交叉验证（cross-validation）是一种评估泛化性能的统计学方法，它比单次划分训练集和测试集的方法更加稳定、全面。\n","* 在交叉验证中，数据被多次划分，并且需要训练多个模型。\n","* 最常用的交叉验证是 k 折交叉验证（k-fold cross-validation），其中 k 是由用户指定的数字，通常取 5 或 10。\n","* 准确率（precision）度量的是被预测为正例的样本中有多少是真正的正例\n","* 召回率（recall）度量的是正类样本中有多少被预测为正类\n","* f-分数是准确率与召回率的调和平均"]},{"cell_type":"markdown","metadata":{"id":"clhTrnn2cQfi"},"source":["【思考】：将上面的概念进一步的理解，大家可以做一下总结"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"fd2teRODcQfi"},"outputs":[],"source":["#思考回答："]},{"cell_type":"markdown","metadata":{"id":"Sf02Ads9cQfj"},"source":["#### 任务一：交叉验证\n","* 用10折交叉验证来评估之前的逻辑回归模型\n","* 计算交叉验证精度的平均值"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"nTRqzAnQcQfj","outputId":"40ab9388-8d9a-4b69-c1b3-e38615259a3c"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAA2kAAACrCAYAAAD8QNSEAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAEXRFWHRTb2Z0d2FyZQBTbmlwYXN0ZV0Xzt0AACAASURBVHic7d1/XBT1vj/w1wKKIiDlovIbjdBElhUpMz0GZRhJnltpiJqgFscISbudsmNXV8vbD+1gHssu2QkyJLI6pR69Fd1V88qRUMlfNwUDQRZ/YKEgKgLz/aPH7hcClkFmdz/rvp6Pxz5gd14zvNf5NNN7Z3ZGJUmSBCIiIiIiIhKCk60LICIiIiIiov+PTRoREREREZFA2KQREREREREJhE0aERERERGRQNikERERERERCYRNGhERERERkUDYpBEREREREQmETRoREd2Q6OhobNiwAQCQk5OD2NhYWdnuqqiogLu7O5qbm29ofiIiInvDJo2IiHps5syZ+OabbxRZVnBwMPLz803PAwMDUV9fD2dnZ0WWT0REJDo2aUREgmlqarJ1CURERGRDbNKIiKyosrISjz76KLy9vTFgwACkpaUhKysL48aNw6JFi3DrrbdCp9OhpaUFr776KoKCgjBw4EDMnj0bFy9eBABcvXoVs2bNwoABA+Dl5YU777wTZ8+eBQBkZWVh6NCh8PDwwJAhQ5CTk9NpLdeuXYOXlxeOHDlieu38+fPo27cvzp07h19//RXx8fHw9vbGLbfcgvj4eJw+fbrDZWVlZWH8+PGm599++y2GDx+O/v37Iy0tDZIkmaadPHkS9913HwYMGAC1Wo2ZM2eitrYWAPDEE0+goqICDz/8MNzd3fHmm2+ivLwcKpXK1LwaDAZMmTIFt956K0JCQvD++++blq3T6fD4449j9uzZ8PDwQFhYGIqKirq7moiIiGyKTRoRkZU0NzcjPj4eQUFBKC8vR1VVFaZPnw4A2LdvH4YOHYpz585hyZIlyMrKQlZWFvR6PX7++WfU19cjLS0NAJCdnY2LFy+isrISFy5cwHvvvYe+ffvi8uXLSE9Px44dO1BXV4e9e/dCq9V2Wo+rqyseffRR5Obmml779NNPce+992LgwIFoaWnBnDlzcOrUKVRUVKBv376mGsypqanBY489hldffRU1NTW47bbb8L//+7+m6ZIk4aWXXoLBYMD//d//obKyEjqdDgCwceNGBAYGYuvWraivr8cLL7zQbvmJiYnw9/eHwWDAZ599hr/85S/47rvvTNO3bNmC6dOno7a2FlOmTJFVMxERkUjYpBERWUlhYSEMBgNWrVqFfv36oU+fPqajT76+vliwYAFcXFzQt29f5OTk4LnnnsPQoUPh7u6O1157DZ988gmamprQq1cvXLhwAaWlpXB2dsbo0aPh6ekJAHBycsKRI0dw5coV+Pj4ICwszGxNM2bMaNOkbdq0CTNmzAAADBgwAI899hjc3Nzg4eGBJUuWYNeuXV2+z+3bt2PEiBGYOnUqevXqhYULF2Lw4MGm6SEhIXjggQfg6uoKb29vPPfcc7KWC/x2JHLPnj1444030KdPH2i1Wjz55JPYuHGjKTN+/Hg89NBDcHZ2xhNPPIEff/xR1rKJiIhEwSaNiMhKKisrERQUBBcXl3bTAgIC2jw3GAwICgoyPQ8KCkJTUxPOnj2LJ554ApMmTcL06dPh6+uLF154AdevX0e/fv2Ql5eH9957Dz4+Ppg8eTJ++uknszXdd999uHLlCvbt24dTp06huLgYjzzyCACgoaEBf/rTnxAUFARPT09MmDABtbW1XV5l0WAwtHk/KpWqzfNz585h+vTp8PPzg6enJ2bNmoWamhqzy2y97FtvvRUeHh5t/m2qqqpMz1s3hG5ubrh69Sq/50dERHaFTRoRkZUEBASgoqKiw4ZBpVK1ee7r64tTp06ZnldUVMDFxQWDBg1Cr169sGzZMhw7dgx79+7Ftm3b8NFHHwEAJk2ahG+//RbV1dUYPnw4nnrqKbM1OTk54fHHH0dubi42bdqE+Ph4UwP01ltv4fjx49i3bx8uXbqE3bt3A0Cb75d1xMfHB5WVlabnkiS1ef7SSy9BpVLh0KFDuHTpEj7++OM2y/z9v8Xv/11++eUX1NXVtfm38fPzM1sTERGRPWGTRkRkJXfddRd8fHywePFiXL58GVevXm3zXa3WEhMTkZGRgbKyMtTX1+Mvf/kLEhIS4OLiAr1ej8OHD6O5uRmenp7o1asXnJ2dcfbsWWzZsgWXL1+Gq6sr3N3dZV22fsaMGcjLy0NOTo7pVEcAqKurQ9++feHl5YVffvkFy5cvl/U+J0+ejKNHj+KLL75AU1MT1q5dizNnzrRZrru7O7y8vFBVVYVVq1a1mX/QoEH4+eefO1x2QEAA7rnnHrz00ku4evUqDh06hA8++AAzZ86UVRsREZE9YJNGRGQlzs7O2Lp1K0pLSxEYGAh/f3/k5eV1mJ07dy6eeOIJTJgwAUOGDEGfPn3wt7/9DQBw5swZTJ06FZ6enrjjjjtw7733YtasWWhpacFbb70FX19f3Hrrrdi1axfefffdLusaM2YM+vXrB4PBgLi4ONPrCxcuxJUrV6BWq3H33XfjwQcflPU+1Wo1Nm/ejMWLF2PAgAEoKSnBuHHjTNOXLVuGAwcOoH///pg8eTIeffTRNvO/9NJLePXVV+Hl5YXVq1e3W35ubi7Ky8vh6+uLRx55BMuXL8cDDzwgqzYiIiJ7oJK6Om+FiIiIiIiIrIZH0oiIiIiIiATCJo2I6CY3f/58uLu7t3vMnz/f1qURERFRB3i6IxERERERkUB4JI2IiIiIiEggbNKIiIiIiIgEwiaNiIiIiIhIIGzSiIiIiIiIBMImjYiIiIiISCBs0oiIiIiIiATiYusCbMHf3x9VVVW2LoOszNfX17Teg4KCUFFRYeOKyNoCAwNx6tQprn8HxjHg2AICAkzrnWPAMRm3AUSic8j7pKlUKuh0ujavlZWVYfPmzZg2bRqGDBnS5TKYt798dnY2jMNdpVLB3NDfuXMnpk2bhs2bNyM6OrrL5TNvH3njelepVFCr1Tavh3nr51uPAeM2wJ7qZ75n+ZiYmC73AyLXz3zP813t/4lEwdMdIWZDwbzyeblE26Ewr2zeSJR6mGeeeeaZt16eyF5YtElbuXIlwsLCoNFooNVqsW/fPrN5nU6H1atXAwCWLl2K/Px8AMCaNWvQ0NDQ4Tzr1q1DSEgIVCoVampqul2jqA0F88rn5RBxh8K8cvnWRKiHeeaZZ5556+aJ7IXFmrSCggJs27YNBw4cwKFDh5Cfn4+AgADZ869YsQITJ04EYL5JGzduHPLz8xEUFNTtGkVuKJi3fl7UHQrzyuW7Q8T6mVcuD0Coephnnnnr5InshcUuHFJdXQ21Wg1XV1cAgFqtNk0LDg5GQkIC9Ho9AGDTpk0ICQlpM39ycjLi4+NhMBhgMBgQExMDtVptmsdo1KhRN1SfaA0C87bNi7xDYV65vFyi1s+8MnkjUephnnnmxckTicJiR9JiY2NRWVmJ0NBQpKamYteuXW2me3p6orCwEGlpaVi4cGGny0lPT4evry/0en27Bq07MjMzERUVhaioKAAQqkFg3rZ50XYQzFsuL4fI9TPf83xrItTDPPPMi5MnEonFmjR3d3fs378fmZmZ8Pb2RkJCArKyskzTExMTTT8LCgosVYZJSkoKioqKUFRUBADCNAjM2zYv2g6CeeaZt2y+O0Ssn3nl8gCEqod52+aJRGPR+6Q5OzsjOjoa0dHRCA8PR3Z2NpKTkwH8dglUo9a/W4sIDQLzts2LtoNgnnnmLZ+XS9T6mVcmbyRKPczbNk8kIosdSTt+/DhKSkpMz4uLi9tc3CMvL8/0c+zYsWaX5eHhgbq6OssU2gnRGgrmlc0D/ASVeeYdMS+HyPUz3/N8ayLUw7xt80SistiRtPr6eixYsAC1tbVwcXFBSEgIMjMzTdOvXbuGMWPGoKWlBbm5uWaXlZKSgri4OPj4+LT7XtratWvx5ptv4syZM9BoNHjooYewYcOGHtUuWkPBvLJ5I1F2EMwzzzzzzFsn312i1c+8snkikakkG9x2PTg4GEVFRW2u+GhNKpUKOp2uw2miNRTMK5fX6XQwDneVSgU5Q1+0HQrzPcsb17vxp63rYd76+d+PAVvXw7x18zExMbL3AyLWz3zP83L3/0S2ZrHTHe2RSA0F85Y7giaXKDsU5plnnnnmlcnLJWr9zCuTJ7IHNjmSZmv+/v6oqqqydRlkZd7e3jh37hwAICgoCBUVFTauiKwtMDAQp06dQmBgICorK21dDtmAcQxwG+CYjOsf4H7AUQUEBHC9k11wyCaNiIiIiIhIVDzdkYiIiIiISCBs0oiIiIiIiATCJo2IiIiIiEggbNKIiIiIiIgEIqtJe/vtt3Hp0iVIkoR58+YhMjIS33zzjaVrIyIiIiIicjiymrS///3v8PT0xDfffIPz58/jww8/xOLFiy1dGxERERERkcOR1aQZr9K/fft2zJkzBxEREbxbOxERERERkQW4yAmNHj0asbGxKCsrw2uvvYa6ujo4Odnv19l4A0vH1PompgMHDsT58+dtXBFZm5+fH06fPs1tgAPjzawdW+sbGXMMOKbW/y9AJDJZN7NuaWlBcXExhg4dCi8vL1y4cAFVVVXQaDTWqFFxKpWq0yOBO3fuxLRp07B582ZER0d3uSzm7Sffer2rVCrodLoul19WVobNmzdj2rRpGDJkCPN2ntfpdJAkyTQWRBqfzFsnb1z3He0H7KF+5nuWj4mJabMfMPe/QCLWz3zP812tdyJRyDoc5uTkhEGDBuHYsWPYvXs3jh49itra2i7nW7lyJcLCwqDRaKDVarFv3z6zeZ1Oh9WrVwMAli5divz8fADAmjVr0NDQ0OE8M2fOxLBhwzBy5EjMnTsX169fl/OWOiTKBoR5y+S7y9YNBfOWzYs2PplnnnnL5+UStX7mlckT2QNZpzu++OKLyMvLw4gRI+Ds7Azgt08iJkyY0Ok8BQUF2LZtGw4cOABXV1fU1NSgsbFRdmErVqww/b5mzRrMmjULbm5u7XIzZ87Exx9/DACYMWMGNmzYgKefflr23zESbQPCvPL57hCtoWBe2TwA4cYn88wzb/l8TEyMUPUwb/08kb2Q1aR9+eWXOH78OFxdXWUvuLq6Gmq12jSPWq02TQsODkZCQgL0ej0AYNOmTQgJCWkzf3JyMuLj42EwGGAwGBATEwO1Wm2ax+ihhx4y/X7XXXfh9OnTsms0Em0Dwrxl8nKJ1lAwr2zeSLTxyTzzzDPPvFgf2BLZkqzTHYcOHdrt0whjY2NRWVmJ0NBQpKamYteuXW2me3p6orCwEGlpaVi4cGGny0lPT4evry/0en27Bq2169evY+PGjXjwwQe7VaeIGxDmLZOXQ7SGgnll862JNj6ZZ5555pkX5wNbIluTdSTNzc0NWq0W999/f5ujaWvXru10Hnd3d+zfvx/ff/899Ho9EhIS8PrrryM5ORkAkJiYaPq5aNGiHryF36SmpmLChAn4wx/+0OH0zMxMZGZmtnlN1A0I87bJi9ZQMK98vjtEG5/MK5sHIFQ9zDPPvHXyck55JRKBrCZtypQpmDJlSrcX7uzsjOjoaERHRyM8PBzZ2dmmJk2lUplyrX+/EcuXL8f58+fxX//1X51mUlJSkJKSYvp7Im9AmLd+XsSGgnnl83KJNj6ZVzZvJEo9zDPPvDh5IlHIatKSkpLQ2NiIEydOAACGDRuGXr16mZ3n+PHjcHJywu233w4AKC4uRlBQkGl6Xl4eFi9ejLy8PIwdO9bssjw8PFBXV9fme21GGzZswNdff43vvvuuW/duE2mDwLxt86I2FMwrn8/Ozu4yL9r4ZF7ZfGsi1MM888yLkycSiayuZufOnbj99tvxzDPPIDU1FaGhodi9e7fZeerr65GUlIQRI0ZAo9Hg2LFjbe5Lde3aNYwZMwZvv/02MjIyzC4rJSUFcXFxHR6inj9/Ps6ePYuxY8dCq9W2uSqkOaJsEJi3bV7khoJ56+dFG5/MK5/vDhHrZ165PMAPbJkna7lw4QK0Wi20Wi0GDx4MPz8/03O5V3+fM2cOjh8/bjbzzjvvICcnR4mS28jPz8e//du/mc0cOHAA//3f/63Y35R1M+vRo0dj06ZNGDZsGADgxIkTSExMxP79+2/ojwYHB6OoqKjDI2PWIPdGhqJtQJjvWf73N7NOSkoSqkFg3vL539/MujVbj0/mrZOvqanpdAzYQ/3M9yz/+/2AXq+3q/qZ73meN7P+jY+PD86cOaPY8gIDA3Hq1ClZWZ1OB3d3dzz//PNtXpckCZIkdevMOGvJz8/HunXr8OWXX3aa2bBhA44cOYI1a9Yo8jdl/Stcv37d1KABQGhoaI9uGm0P7GmDw/yNfWImekPBvPXyoo1P5i2Xl0Pk+pnveb41Eeph3rZ5R3XmzBlTU2TuodfrTbfAMperqKi4oTpKS0sxcuRIzJ8/H5GRkaiurkZKSgqioqIQFhbW5gy58ePHo7i4GE1NTfDy8sLixYsRERGBsWPH4ty5cwCAl19+2dQkjR8/HosXL8Zdd92FYcOGYe/evQCAy5cv47HHHkNERAQSExMRFRWF4uLidrX985//xLBhwzB+/Hh89dVXptf/9a9/YezYsRg1ahTGjRuHkpISXLlyBStWrEBOTg60Wi0+++yzDnPdIskwZ84cae7cuZJer5f0er305JNPSsnJyXJmFVJgYKAEgA8HewQGBprGgLe3t83r4cP6Dz8/P24DHPxh3A5wDDjmIyAggP8v4OCP1v8v4MiArlsAvV4vqdVqSa/XK7I8o2XLlkmrVq2SJEmSSkpKJJVKJRUWFpqmX7hwQZIkSbp+/bo0fvx46ejRo5IkSdK4ceOkgwcPStevX5cASNu3b5ckSZIWLVokvfbaa5IkSdKSJUukjIwMU/6FF16QJEmSvvrqK2nSpEmSJEnSa6+9JqWmpkqSJEnFxcWSk5OTdPDgwTY1Xr58WfLz85NKS0ullpYW6dFHH5X++Mc/SpIkSbW1tVJTU5MkSZK0Y8cO6fHHH5ckSZLef/996dlnnzUto7OcXLIuHLJ+/Xq88847WLt2LSRJwoQJE5CamipnViHJPRxLNy/jJy7kmLgNII4B4hgg6pw1j0jedtttuPPOO03Pc3Nz8cEHH6CpqQkGgwHHjh3DiBEj2szTt29fxMXFAfjta1nff/99h8t+9NFHTZny8nIAwJ49e/Diiy8CACIiIhAWFtZuvmPHjiE0NBS33XYbAGDmzJn46KOPAAC1tbWYPXs2Tp48afZ9yc11RlaT5urqiueeew7PPffcDf0RIiIiIiIS342cYtoT/fr1M/1eUlKCt99+G4WFhfDy8sKsWbNw9erVdvP07t3b9LuzszOampo6XLbx/s6tM5LM7yR2douwJUuWYNKkSUhNTUVpaSkefPDBHuU6Y/Y7aY8//jgAIDw8HBqNpt2DiIiIiIhuDjf6HUClXLp0CR4eHvD09ER1dTW+/vprxZZtNH78eHz66acAgMOHD+PYsWPtMiNGjMCJEydQVlYGSZKQm5trmnbx4kX4+fkBALKyskyvG28Z1lVOLrNH0t5++20AwLZt27q9YCIiIiIisg89uUhLR7fJuhGRkZEYMWIERo4ciaFDh2LcuHGKLLe1BQsWYPbs2dBoNIiMjMTIkSPRv3//Nhk3Nze89957iIuLg1qtxrhx40yX/3/xxRcxd+5cvPnmm23e93333YdVq1Zh1KhRWLJkSac5uWRdgv/FF1/EG2+80eVrREREREQktqCgoBu+ImNHunMJfltrampCU1MT+vTpg5KSEsTGxqKkpAQuLrK+BWY1spq0yMhIHDhwoM1rGo0Ghw4dslhhRERERERESqqtrcX999+PpqYmSJKE1atXIzY21tZltWO2ZVy/fj3effdd/Pzzz22+g1ZXV2eRw49ERERERESW4uXlhf3799u6jC6ZPZJ28eJF/Prrr3jppZfw+uuvm1738PDArbfeapUCiYiIiIiIHIms0x2Nzp071+YymIGBgRYpytJ8fHxw5swZW5dBVtb6fGmlz8Um+2AcA/7+/qiqqrJ1OWQD3t7eOHfuHLcBDqr1fiAwMBCVlZU2roiszZ6+O0WOTVaTtnXrVjz33HMwGAwYOHAgTp06hTvuuANHjx61Ro2KU6lUsu6R0JOr3DAvXr71elepVNDr9XZVP/M9zxvHgEqlgk6na5MvKyvD5s2bMW3aNAwZMqTL5TNvn/mGhgbTGDC3H7CH8cx89/Md7QfsqX7me56X+/+ARLZm9j5pRi+//DL+9a9/ITQ0FGVlZfjuu+9u+u+k2dMGh/nu5wEIVQ/zts2L2lAwr3xeDtHGJ/PK5lsToR7mbZsnEpWsJq1Xr14YMGAAWlpa0NLSgpiYGBQXF3c538qVKxEWFgaNRgOtVot9+/aZzet0OqxevRoAsHTpUuTn5wMA1qxZg4aGhg7nmTdvHiIiIqDRaDB16lTU19fLeUtmibYBYV7ZvJEo9TBv27zIDQXz1s+LNj6ZVzbfXaLVz7yyebKeCxcuQKvVQqvVYvDgwfDz8zM9b2xslL2cv//977K+slRaWgqtVms28/PPP+OTTz6R/betTVaT5uXlhfr6ekyYMAEzZ87Es88+2+W9BAoKCrBt2zYcOHAAhw4dQn5+PgICAmQXtmLFCkycOBGA+SYtIyMDP/74Iw4dOoTAwECsW7dO9t/oiGgbEOaVzbcmQj3M2zYvWoPAvG3zoo1P5pXPd4eI9TOvXN7R+fv7Q6VSKfbw8/Mz+/cGDBiA4uJiFBcXY/78+Vi0aJHpee/evWXXLbdJk0P0Jk3WXdu++uor9O3bFxkZGcjJycHFixexdOlSs/NUV1dDrVbD1dUVAKBWq03TgoODkZCQAL1eDwDYtGkTQkJC2syfnJyM+Ph4GAwGGAwGxMTEQK1Wm+Yx8vT0BABIkoQrV65ApVLJeUsdEm0Dwryy+e4SrX7mlc2L1iAwb9u8aOOTecvk5RK1fuaVyRNQVVXV7rvZRjeyvc3Ozr7hWrKzs/HOO++gsbER99xzD9atW4eWlhbMmTMHxcXFkCQJKSkpGDRoEIqLi5GQkIC+ffuisLCwTYP3ww8/YN68eejXr1+br2WdPHkSycnJqK+vh5OTE959912MGTMGixcvRklJCbRaLebOnYvJkyd3mLOVLo+kNTc3449//COcnJzg4uKCpKQkpKenY8CAAWbni42NRWVlJUJDQ5Gamopdu3a1me7p6YnCwkKkpaVh4cKFnS4nPT0dvr6+0Ov17Ro0ozlz5mDw4MH46aefsGDBgg4zmZmZiIqKQlRUVIfTRduAMK98vjtErJ955fIAhGoQmLdtXrTxybzl8nKIXD/zPc+TeTe6vb1RR44cwT/+8Q/s3bsXxcXFaGpqwieffIL9+/ejpqYGhw8fxpEjRzB79mwkJCRAq9UiLy+vwyNwycnJWL9+PQoKCtDc3Gx63cfHB99++y0OHjyInJwcpKenAwBef/1101e40tPTO83ZSpdNmrOzM9zc3HDx4sVuLdjd3R379+9HZmYmvL29kZCQgKysLNP0xMRE08+CgoLuVf07H374IQwGA+644w7k5eV1mElJSUFRURGKioraTRNtA8K8ZfJyiVo/88rkjURpEJi3bV608ck888yL84Gto7H0RZk6kp+fjx9++AFRUVHQarXYtWsXTp48iZCQEBw/fhzPPvssvv76a/Tv39/scmpqanDlyhXTEbQnnnjCNO3atWuYN28eRo4cienTp+PYsWMdLkNuzlpkne7Yp08fhIeH44EHHkC/fv1Mr69du9bsfM7OzoiOjkZ0dDTCw8ORnZ2N5ORkAGhzWmJPTlFs/bcSEhKwatUqzJkzR/Z8Im5AmLdMPiYmRqh6mLd+vjURGgTmbZsXbXwyzzzz4nxg62gsvb3tjCRJmDt3Ll555ZV20w4dOoQdO3Zg7dq1+Pzzz5GZmWl2WZ31E2+99RYCAgLw8ccf4/r163B3d+9Rzlq6PJIGAJMnT8Yrr7yCCRMmYPTo0aaHOcePH0dJSYnpeXFxMYKCgkzPjUe88vLyMHbsWLPL8vDwQF1dXbvXJUlCaWmp6fetW7di+PDhct4SAHE3IMwzz7xl8t0hWkPBvLJ5AMKNT+aZZ97yeWrPVg0aAEycOBGffvopampqAPx2FciKigqcP38ekiRh2rRpWL58OQ4cOACg855ArVajT58+prPzcnJyTNMuXrwIHx8fqFQqZGdnm+6T9/tldZazFVlH0pKSknDlyhVUVFRg2LBhshZcX1+PBQsWoLa2Fi4uLggJCWnTAV+7dg1jxoxBS0sLcnNzzS4rJSUFcXFx8PHxafO9NEmSkJSUhEuXLkGSJERERGD9+vWy6hN5A8I888xbJi+XaA0F88rmjUQbn8wzz7zt847Glg0aAISHh2PZsmWYOHEiWlpa0KtXL7z33ntwdnbGvHnzIEkSVCoV3njjDQC/XYfiySef7PDCIR9++CGefPJJ9OvXD7GxsabX09LSMHXqVOTm5mLixImmixqOGjUKzc3NiIiIwLx58zrN2YpKktEmbt26Fc8//zwaGxtRVlaG4uJiLF26FFu2bLmhPxocHIyioqI2V3y0JpVKBbVaLcwGgXnr5FUqlelTkda/20v9zPc8HxMTY9rgK3lVK+btJ6/T6UxjQM6npCKPZ+a7n//9fkCv19tV/cz3PC/3v/2bnb+/P6qqqhRbnp+fH06fPq3Y8kjm6Y46nQ6FhYXw8vICAGi1WpSVlVm0MEsTeQPCPPPM2yYvWkPBvPL57hBtfDKvbB6AUPUwb9u8ozl9+jQkSVLswQZNebKOpI0ZMwb79u3DqFGjcPDgQQCARqPBoUOHLF6gJQQGBqKystLWZZCVBQYG4tSpUwCAoKAgVFRU2LgisjbjGFD6E0SyH76+vqiqquI2wEG13g/4+PgodlNcsh+txwCRyGR9J23kyJHYtGkTmpubUVJSgrVr1+Kee+6xdG0Wwx0zcQPt2PiJH3EbKSHdsQAAFDZJREFUQNXV1bYugYioU7JOd/zb3/6Go0ePwtXVFTNmzED//v3x9ttvW7o2IiIiIiIihyPrdEfjOf9dvUZEREREREQ9I6tJi4yMNN2fwNxrRERERERE1DNmv5O2Y8cObN++HVVVVUhPTze9funSJbi4yPo6GxEREREREXWD2U7L19cXUVFR2LJlC0aPHm163cPDAxkZGRYvjoiIiIiIyNHIOt2xqamJR86IiIiIiIiswGyT9vjjj+PTTz9FeHg4VCpVu+n2ep803h/HMfE+aTR48GBUV1dz/Tsw43bAz88PBoPB1uWQlfn5+ZluwcHtgGPifdLIXpht0qqrq+Hj49PpYA4KCrJYYZakUqmg1+uFutM985bPq1QqGId769/tpX7me56vqamBJEnt1r+91M98z/PGda9SqZCUlIQhQ4Z0ufyysjLTFY2Zt++8Tqdrtx8QaXwyb/l8V9t/IlGYPYfRx8cHgP02Y+aIvAFhnnnmLZOPiYkRqh7mrZ9vTfSGgnnL50Ubn8zbNk8kElk3s75RK1euRFhYGDQaDbRaLfbt22c2r9PpsHr1agDA0qVLkZ+fDwBYs2YNGhoazM67YMECuLu7y65NlA0C88wzzzzz1st3h2gNBfPK5gF+YMs8kbgsdjWQgoICbNu2DQcOHICrqytqamrQ2Ngoe/4VK1aYfl+zZg1mzZoFNze3DrNFRUWora3tVn0ibBCYZ5555pm3bl4u0RoK5pXNG4k2Ppm3TZ5IRGaPpK1evRqVlZU3tODq6mqo1Wq4uroCANRqNXx9fQEAwcHBePHFF3HXXXfhrrvuQmlpabv5k5OT8dlnn2Ht2rUwGAyIiYnp8FSl5uZm/PnPf8abb755Q3V2RrQNCPPK5gF+gso8846Yl0O0hoJ5ZfOtiTY+mbd+nkhUZpu0qqoq3HPPPZgwYQLWr1+Pmpoa2QuOjY1FZWUlQkNDkZqail27drWZ7unpicLCQqSlpWHhwoWdLic9PR2+vr7Q6/XQ6/Xtpq9btw5TpkwxfX9OCaJtQJhXNm8kSj3MM8+8OHnRGgrmlc13l2jjk3ll80QiM9ukZWRkoKKiAq+88goOHToEjUaDuLg4fPTRR6irqzO7YHd3d+zfvx+ZmZnw9vZGQkICsrKyTNMTExNNPwsKCm6oeIPBgM2bN2PBggVdZjMzMxEVFYWoqCizOdE2IMwrm29NhHqYZ555cfKiNRTMK5/vDtHGJ/PK5olEZ7ZJA367RO29996L9evXo7KyEgsXLkRGRgYGDRrU5cKdnZ0RHR2N5cuXY926dfj888/bLLej37vj4MGDKC0tRUhICIKDg9HQ0ICQkJAOsykpKSgqKkJRUVGnyxNtA8K8svnuEq1+5plnng0a8z3LyyXa+GRe2TyRPeiySTM6fPgwli5dimeeeQa9e/fGf/7nf5rNHz9+HCUlJabnxcXFbS7ln5eXZ/o5duxYs8vy8PDo8Mjd5MmTcebMGZSXl6O8vBxubm4dfr9NDtE2IMwrn+8OEetnXrk8AKHqYd62eVEbCuaVz8sh2vhkXtk8kb0we3XHkpIS5Obm4pNPPoGzszOmT5+Ob775BkOHDu1ywfX19ViwYAFqa2vh4uKCkJAQZGZmmqZfu3YNY8aMQUtLC3Jzc80uKyUlBXFxcfDx8enwe2k9JdoGhHnL5OUStX7mlckbiVIP87bNi9xQMG/9vGjjk3nl80T2QiWZue360KFDkZiYiOnTpyM8PFyxPxocHIyioiKo1WrFltkdre82L+IGhHnL5GNiYkzrvfUYsJf6me953rjeO1v/otfPfM/zrcdAUlKSUA0C85bP63S6TvcDIoxP5i2fr6mpkbX9J7I1s6c7fv3114iLi2vXoH3//fc4efKkRQuzBlE3IMwzz7xl8t0hYv3MK5cHIHxDwbz18qKNT+YtlyeyG5IZkydPln788cd2r//www9SfHy8uVmFFhgYKAHgw8EegYGBHAMO/ggICOD6d/CHcTvg7e1t81r4sP7Dz8+P+wEHf7T+fwEikZk93XHkyJE4cuRIh9PCw8Nx+PDhzmYlIiIiIiKiG2D2dMerV692Ou3KlSuKF0NEREREROTozDZpd955J95///12r3/wwQcYPXq0xYoiIiIiIiJyVGZPdzx79iweeeQR9O7d29SUFRUVobGxEf/4xz8wePBgqxVKRERERETkCMw2aUZ6vd703bSwsDDcd999Fi+MiIiIiIjIEclq0oiIiIiIiMg6zH4njYiIiIiIiKyLTRoREREREZFAXGxdgC0EBQWhoqLC1mWQlQUEBJjWO8eAYwoMDMSpU6fg4+ODM2fO2LocsgHjGOA2wDEZ1z8A+Pv7o6qqysYVkbV5e3vj3Llzti6DqEsO+Z00lUoF49veuXMnpk2bhs2bNyM6OrrLeZm333xMTIxpvbceA/ZSP/M9zxvXe2frX/T6me95vvUY0Ov1Nq+HeevmW/+3r1KpoNPpTPmysjJs3rwZ06ZNw5AhQ7pcPvP2mW9oaJC1/SeyNYc+3dEedijMM8+8cvnuELF+5pXLAxCqHuZtmxe1oWBe+TyRvbBok7Zy5UqEhYVBo9FAq9Vi3759ZvM6nQ6rV68GACxduhT5+fkAgDVr1qChoaHDeZKTkzFkyBBotVpotVoUFxfLqk20HQTzzDNv+bxcotbPvDJ5I1HqYd62eZEbCuatnycShcW+k1ZQUIBt27bhwIEDcHV1RU1NDRobG2XPv2LFCtPva9aswaxZs+Dm5tZhdtWqVZg6dWq36hNpB8E888xbJx8TEyNUPcxbP9+aCPUwb9u8aA0C87bNE4nEYk1adXU11Go1XF1dAQBqtdo0LTg4GAkJCdDr9QCATZs2ISQkpM38ycnJiI+Ph8FggMFgQExMDNRqtWmenhJlB8E888wzz7z18t0hYv3MK5cHIFSDwLxt80SisdjpjrGxsaisrERoaChSU1Oxa9euNtM9PT1RWFiItLQ0LFy4sNPlpKenw9fXF3q9vtMGbcmSJdBoNFi0aBGuXbvWYSYzMxNRUVGIiooCwE9QmWeeeeYdMS+XqPUzr0zeSJQGgXnb5olEZLEmzd3dHfv370dmZia8vb2RkJCArKws0/TExETTz4KCghv+O6+99hp++ukn/PDDD/jll1/wxhtvdJhLSUlBUVERioqKZC1XtB0K88rmAQhVD/PMM2+dvBwi1898z/OtidAgMG/bPJGoLNakAYCzszOio6OxfPlyrFu3Dp9//rlpmkql6vD37vLx8YFKpYKrqyvmzJmDwsLCHtUMiLdDYV7ZvJEo9TDPPPPMM2+dfHeJ1lAwr2yeSGQWa9KOHz+OkpIS0/Pi4mIEBQWZnufl5Zl+jh071uyyPDw8UFdX1+G06upqAIAkSfjyyy8xcuTIHtUt2g6FeWXzrYlQD/PMM88889bLd4doDQXzyuaJRGexC4fU19djwYIFqK2thYuLC0JCQpCZmWmafu3aNYwZMwYtLS3Izc01u6yUlBTExcXBx8en3ffSZs6cifPnz0OSJGi1Wrz33ns3XLOIOxTmlct3l2j1M88888wz37O8XKI1FMwrmyeyByrJBrddDw4ORlFRUZsrPlqTSqVqd7d5UXcozCuXj4mJMa33jsaA6PUz3/O8cb2rVCqo1Wqb18O89fOtx4BxG2BP9TPfs/zv9wM6na5dXrSGgnll8zqdzuz+n0gUFjvd0Z6IvENhXrm8XKLWz7wyeSNR6mGeeebFyYvWUDCvfJ7IXtjkSJqtBQUFoaKiwtZlkJUFBgbi1KlTADgGHFVAQAAqKioQGBiIyspKW5dDNmDcDnAb4Jha7wf8/f1RVVVl44rI2nx9fbneyS44ZJNGREREREQkKp7uSEREREREJBA2aURERERERAJhk0ZERERERCQQNmkOqPX96sgxcQwQxwBxDDg2rn8isbFJc0DcMBPHAHEMEMeAY+P6JxIbmzQiIiIiIiKBsEkjIiIiIiISiLNOp9PZugiyvtGjR9u6BLIxjgHiGCCOAcfG9U8kLt7MmoiIiIiISCA83ZGIiIiIiEggbNLsmLOzM7RarelRXl7eaXbnzp2Ij4/vcFpwcDBqamravb5kyRIEBATA3d1dqZJJYZYcAw0NDZg8eTKGDx+OsLAwLF68WMnSSSGW3g48+OCDiIiIQFhYGObPn4/m5malSicFWHr9G02ZMgUjR47sablkAZYeA9HR0Rg2bJhp+efOnVOqdCIyw8XWBdCN69u3L4qLiy22/IcffhhpaWm4/fbbLfY3qGcsPQaef/55xMTEoLGxEffffz927NiBuLg4i/096j5Lj4FPP/0Unp6ekCQJU6dOxebNmzF9+nSL/T3qHkuvfwD44osv+GGdwKwxBnJychAVFWXRv0FEbfFI2k3m6tWrmDNnDsLDwzFq1Cjo9fp2mQsXLiA2NhajRo3Cn/70J3T2tcS7774bPj4+li6ZFKbUGHBzc0NMTAwAoHfv3oiMjMTp06ctXj/1nJLbAU9PTwBAU1MTGhsboVKpLFo79ZyS67++vh5//etf8fLLL1u6bFKQkmOAiGyDTZodu3Lliun0g0ceeQQA8M477wAADh8+jNzcXCQlJeHq1att5lu+fDnGjx+PgwcPYsqUKaioqLB67aQMa42B2tpabN26Fffff79l3gjdMGuMgUmTJmHgwIHw8PDA1KlTLfdmqNssvf7/4z/+A//+7/8ONzc3y74RumHW2AbMmTMHWq0Wr7zyCps5Iivh6Y52rKNTHPbs2YMFCxYAAIYPH46goCCcOHGiTWb37t344osvAACTJ0/GLbfcYp2CSXHWGANNTU1ITExEeno6hg4dqvA7oJ6yxhj4+uuvcfXqVcycORP/8z//gwceeEDhd0E3ypLrv7i4GKWlpcjIyDD7PSeyLUtvA3JycuDn54e6ujo89thj2LhxI2bPnm2Bd0JErfFI2k1G7idcPGXp5qX0GEhJScHtt9+OhQsX9qQssiJLbAf69OmDKVOm4KuvvrrRsshKlFr/BQUF2L9/P4KDgzF+/HicOHEC0dHRClRIlqbkNsDPzw8A4OHhgRkzZqCwsLBHtRGRPGzSbjITJkxATk4OAODEiROoqKjAsGHDOs3s2LEDv/76q9XrJMtRcgy8/PLLuHjxItasWWPZoklRSo2B+vp6VFdXA/jtiOr27dsxfPhwC1dPPaXU+n/66adhMBhQXl6OPXv2IDQ0FDt37rR4/dRzSo2BpqYm0xUfr1+/jm3btvEqn0RWwibtJpOamorm5maEh4cjISEBWVlZcHV1bZNZtmwZdu/ejcjISHzzzTcIDAzscFkvvPAC/P390dDQAH9/f+h0Oiu8A+oppcbA6dOnsXLlShw7dgyRkZHQarXYsGGDtd4G9YBSY+Dy5cuYMmUKNBoNIiIiMHDgQMyfP99ab4NukJL7AbJPSo2Ba9euYdKkSdBoNNBqtfDz88NTTz1lrbdB5NBUEr8BSkREREREJAweSSMiIiIiIhIImzQiIiIiIiKBsEkjIiIiIiISCJs0IiIiIiIigbBJIyIiIiIiEgibNCIiO+Xs7AytVouwsDBERETgr3/9K1paWszOU15ejk2bNlm8tieffBLHjh0zm/nyyy+7zBARETkiNmlERHaqb9++KC4uxtGjR/Htt99i+/btWL58udl5rNWkbdiwASNGjDCbYZNGRETUMTZpREQ3gYEDByIzMxPr1q2DJEkoLy/HH/7wB0RGRiIyMhJ79+4FACxevBjff/89tFotMjIyOs21Vl5ejuHDhyMpKQkajQZTp05FQ0MDAOC7777DqFGjEB4ejrlz5+LatWsAgOjoaBQVFQEA3N3dsWTJEkRERODuu+/G2bNnsXfvXmzZsgV//vOfodVqcfLkSaxduxYjRoyARqPB9OnTrfQvR0REJB7ezJqIyE65u7ujvr6+zWu33HILfvrpJ3h4eMDJyQl9+vRBSUkJEhMTUVRUhJ07d2L16tXYtm0bAKChoaHDXGvl5eUYMmQI9uzZg3HjxmHu3LkYMWIE0tLScPvtt+O7775DaGgoZs+ejcjISCxcuBDR0dFYvXo1oqKioFKpsGXLFjz88MN44YUX4OnpiZdffhnJycmIj4/H1KlTAQC+vr4oKyuDq6sramtr4eXlZZ1/SCIiIsHwSBoR0U3E+Lnb9evX8dRTTyE8PBzTpk3r9LRCubmAgACMGzcOADBr1izs2bMHx48fx5AhQxAaGgoASEpKwu7du9vN27t3b8THxwMARo8ejfLy8g7/hkajwcyZM/Hxxx/DxcWlW++biIjoZsImjYjoJvHzzz/D2dkZAwcOREZGBgYNGoQff/wRRUVFaGxs7HAeuTmVStXuudwTMXr16mWa39nZGU1NTR3m/vnPf+KZZ57B/v37MXr06E5zRERENzs2aUREN4Hz589j/vz5SEtLg0qlwsWLF+Hj4wMnJyds3LgRzc3NAAAPDw/U1dWZ5uss93sVFRUoKCgAAOTm5mL8+PEYPnw4ysvLUVpaCgDYuHEj7r33Xtk1t66lpaUFlZWViImJwZtvvona2tp2p3ISERE5CjZpRER26sqVK6ZL8E+cOBGxsbFYtmwZACA1NRXZ2dm4++67ceLECfTr1w/Ab6cUuri4ICIiAhkZGZ3mfu+OO+5AdnY2NBoNfvnlFzz99NPo06cPPvzwQ0ybNg3h4eFwcnLC/PnzZdc/ffp0rFq1CqNGjUJJSQlmzZqF8PBwjBo1CosWLeJ30oiIyGHxwiFERGRWeXk54uPjceTIEVuXQkRE5BB4JI2IiIiIiEggPJJGREREREQkEB5JIyIiIiIiEgibNCIiIiIiIoGwSSMiIiIiIhIImzQiIiIiIiKBsEkjIiIiIiISCJs0IiIiIiIigbBJIyIiIiIiEsj/A1pgE6zaHppKAAAAAElFTkSuQmCC\n","text/plain":["<IPython.core.display.Image object>"]},"execution_count":6,"metadata":{},"output_type":"execute_result"}],"source":["#提示：交叉验证\n","Image('Snipaste_2020-01-05_16-37-56.png')"]},{"cell_type":"markdown","metadata":{"id":"fJqtB0VQcQfl"},"source":["#### 提示4\n","* 交叉验证在sklearn中的模块为`sklearn.model_selection`\n","\n","https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_score.html?highlight=cross_val_score#sklearn.model_selection.cross_val_score"]},{"cell_type":"code","execution_count":12,"metadata":{"id":"wzlPn7SWcQfl","executionInfo":{"status":"ok","timestamp":1680522290759,"user_tz":-540,"elapsed":359,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["#写入代码\n","from sklearn.model_selection import cross_val_score"]},{"cell_type":"code","execution_count":13,"metadata":{"id":"FK4WbN71cQfl","executionInfo":{"status":"ok","timestamp":1680522294833,"user_tz":-540,"elapsed":2236,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["#写入代码\n","rfc = RandomForestClassifier(n_estimators=100, max_depth=5)\n","scores = cross_val_score(rfc, X_train, y_train, cv=10)#确定交叉验证拆分策略。cv=None,使用默认的5折交叉验证"]},{"cell_type":"code","execution_count":14,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"s5p2afoscQfm","executionInfo":{"status":"ok","timestamp":1680522297420,"user_tz":-540,"elapsed":352,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"385ae986-bd07-41a6-b973-0ebcc6f424d0"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["array([0.85074627, 0.79104478, 0.85074627, 0.7761194 , 0.82089552,\n","       0.8358209 , 0.79104478, 0.7761194 , 0.77272727, 0.77272727])"]},"metadata":{},"execution_count":14}],"source":["#写入代码\n","#k折交叉验证分数\n","scores"]},{"cell_type":"code","execution_count":15,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"zFKNQf1acQfm","executionInfo":{"status":"ok","timestamp":1680522301072,"user_tz":-540,"elapsed":663,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"f4674ef4-0cde-4fbb-9258-c70e7414016c"},"outputs":[{"output_type":"stream","name":"stdout","text":["Average cross-validation score:0.80\n"]}],"source":["#写入代码\n","#平均交叉验证分数\n","print(\"Average cross-validation score:{:.2f}\".format(scores.mean()))"]},{"cell_type":"markdown","metadata":{"id":"6eUFvZ8ucQfm"},"source":["#### 思考4\n","* k折越多的情况下会带来什么样的影响？"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"O3PTWxNhcQfm"},"outputs":[],"source":["#思考回答\n","'''\n","答：k折越多，每次训练时的训练集数据越多，测试集数据越少。\n","数据充分的训练集拟合效果可能越好，且训练次数越多，评估结果的稳定性和保真性越高\n","此外，折数过多意味着训练的次数多，且数据量大的情况下单次训练的时间也会变长，即复杂度会变高。\n","'''"]},{"cell_type":"markdown","metadata":{"id":"eeX8uhjmcQfn"},"source":["#### 任务二：混淆矩阵\n","* 计算二分类问题的混淆矩阵\n","* 计算精确率、召回率以及f-分数"]},{"cell_type":"markdown","metadata":{"id":"Qp5ieb73cQfn"},"source":["【思考】什么是二分类问题的混淆矩阵，理解这个概念，知道它主要是运算到什么任务中的"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"v0NKNiCDcQfn"},"outputs":[],"source":["#思考回答\n","'''\n","答：这个在之前【吃瓜教程】中学习过，\n","具体看https://blog.csdn.net/sinat_33209811/article/details/125755163\n","'''"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"0BwaLcO7cQfo","outputId":"99bf92e5-3a89-498a-89da-63c8ece6c0fa"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAdQAAAECCAYAAABHQCZeAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAEXRFWHRTb2Z0d2FyZQBTbmlwYXN0ZV0Xzt0AACAASURBVHic7d15fEz3/j/wVxJZJosI2RCClNibSmqvRFAluBSRKk3Qa2vdIpYKkSDE3nSzu4nitiEESQRVwlVpLaUtbtHa19jXhETevz/8zLcjiZnIzJxJvJ6Pxzw8MvOec14zOeadc+ZzzsdMRARERERUIuZKByAiIioL2FCJiIj0gA2ViIhID9hQiYiI9IANlYiISA/YUImIiPSADZWIiEgP2FCJiIj0gA2ViIhID9hQiYiI9KCc0gHIeJydnVGjRg2lYxAVcPz4cQCAt7e3wkmICjp79iyuXbumtY4N9RVSo0YNHDhwQOkYRAUEBAQAADIyMhTNQVQYPz8/nep4yJeIiEgPuIdKRIqbNGmS0hGISowNlYgU1759e6UjEJUYD/kSkeIOHz6Mw4cPKx2DqES4h0pEihs5ciQADkqi0o17qERERHrAhkpERKQHbKhERER6wIZKRESkBxyURESKmzFjhtIRiEqMDZWIFNeyZUulIxCVGA/5EpHi9u7di7179yodg6hEuIdKRIqLiIgAwPNQqXRjQyWdZGRkoG3btkrHMLr4+HiEhYW9sMbMzKzYyx06dCgWLlz4kqlKLiEhAQMGDNBat3PnTvVMMPR/zpw5g5o1ayodQyt/f/+X/iNl2bJl2LNnj34D4en/F2tra1hbW8PW1hYuLi5wc3ODh4cHGjRoAGdnZ72v01jYUIkUsHz5cowZMwZeXl5KRyEq1J49e7BixQqjr9fV1RVNmzZFQEAA3n77bTRq1MjoGV4Wv0MlUkBubi4iIyOVjkFkcrKyspCamooxY8agcePGaNCgAWbOnIlbt24pHU0rNlQihXz33Xf49ddflY5BZNKOHTuGCRMmoHr16hg3bhzu3bundKQisaESKUREMHHiRKVjmIS4uDjExcUpHYNM2P379zFnzhzUrVsXycnJSscpFBsqkYLS0tIMMvCjtPHx8YGPj4/SMagUuHTpEt59912Eh4cjLy9P6Tga2FCJFDZhwgSlIyhu+/bt2L59u9IxqBSZP38++vbta1JNlQ2VSGF79uxBWlqa0jEUFRMTg5iYGKVjUCmzdu1arae1GRMbKukkICAAImLwW3x8vE55oqKijJLHWP9ZJ06cCBExyrrIePz9/Y2ynRZ1M8aFMnbu3FmsTPn5+bh16xb+/PNP7Nu3D8uWLcMHH3wAT0/Pl1r/6tWrTeb7dzZUIhPw66+/4rvvvlM6BpHBmZmZoUKFCvDy8sKbb76JQYMGYcWKFThz5gzS0tLw1ltvFXuZ48aNwx9//GGAtMXDhkpkIiIjI5Gbm6t0DCLFdO7cGbt378batWvh4OCg8/Nyc3MxYsQIAybTDRsqkYn466+/sHz5cqVjECmuV69e2LdvH1577TWdn7N9+3bFR8yzoRIZQbt27WBjY6O1burUqcjOzjZCItOyePFiLF68WOkYZELq1q2L1NRUlC9fXufnzJs3z4CJtGNDJTICDw8PDB8+XGvd5cuX8cUXXxghkWnx9vaGt7e30jHIxHh7e2PlypU616elpSl6iUI2VCIjiYiI0Omv7VmzZuH27dtGSGQ6UlJSkJKSonQMMkHdunVDYGCgTrW5ubmKbkdsqERGUqlSJYSHh2utu3XrFmbPnm2ERKZj3rx5ih+uI9M1adIknWt37NhhwCQvxoZKZESjR4+Gi4uL1rrPP/8cV65cMUIiItPXtm1b1K5dW6fan3/+2cBpisaGSmRE9vb2Ol0Q/+HDh5g2bZoREhGVDrpOdH/8+HE8evTIsGGKwIZKZGRDhw7V6aowS5cuxalTp4yQiMj0tW3bVqc6EcH58+cNnKZwbKhERmZtbY3o6Gitdbm5uYiKijJ8IKJSoDijwNlQiV4h/fv3R/369bXW/ec//8Hvv/9uhETKWrlyZbFOj6BXT6VKlXSuvXv3rgGTFI0NlUgBFhYWOs2ukp+f/0pMQl6tWjVUq1ZN6RhkworTUB8+fGjAJEVjQyVSSI8ePdC0aVOtdSkpKdi7d68REiknMTERiYmJSscgE2ZlZaVz7ZMnTwyYpGhsqEQKio2N1amurE9CvnDhQixcuFDpGGTCbty4oXOtra2tAZMUjQ2VSEGBgYFo37691rrdu3cjPT3dCImITBMbKhFppeteakREBCchp1fWxYsXda51d3c3YJKisaESKczPzw89e/bUWnf48GGsWbPGCImITE9xpmbT5TxvQ2BDJTIBMTExsLCw0FoXGRmJvLw8IyQiMi0ZGRk61Tk7O8PJycmwYYrAhkpkAurWrYsPPvhAa93Jkyfx73//2wiJjCspKQlJSUlKxyATdenSJZ2v0avLyHlDYUMlMhFTpkyBtbW11rqyOAm5s7MznJ2dlY5BJmrOnDnIzc3VqbZFixYGTlM0NlQiE1GtWjUMGzZMa93Fixfx1VdfGSGR8SQkJCAhIUHpGHq3a9cumJmZGfUWFham9MvWq8uXL2PJkiU61//jH/8wYJoXY0MlMiETJ06Eg4OD1rqZM2fizp07RkhkHGW1oVLJPH78GL169dL5ykd16tRBo0aNDJyqaGyoRCbE2dkZo0eP1lp38+ZNzJkzxwiJiJSRn5+PYcOGFesqYSNGjDBgIu3YUIlMTHh4uE7fJ8bFxeHq1atGSERkXDdv3kRQUFCxBuC5uLhg4MCBBkylXaluqLNnzy50KLWZmZnJfMcUFhYGPz8/pWNQKeLg4ICIiAitdQ8ePNDpAvtEpUVubi7i4+PxxhtvYMuWLcV67syZMxW7QtIzZbKhZmZmonfv3sYPRKQnw4cP12n2lSVLluDMmTOGD0RkIDk5OcjIyEB0dDRq1aqFgQMH4ty5c8VaRkBAAAYMGGCghLorp3QAQ2jevLnSEYhK5Nkk5IMGDXph3ePHjxEVFYUVK1YYKZlhbN68WekI9JJmzpxZrAFlIoK7d+/ixo0buHnzJk6ePInHjx+/9PqrVKmC7777DmZmZi+9DL0RHYSGhoqvr69s27ZNGjVqJLa2ttKqVSs5cuSIRt2TJ08kNjZWvLy8xMrKSmrXri0JCQkaNfn5+TJp0iRxcXERBwcHGTBggHz77bcCQE6fPq2uGz9+vDRs2FDs7OykatWq0rdvX7l8+bL6cU9PTwGgcdu5c6fI04udypdffikiIpMnTxY3Nzd58uSJRo6UlBQBICdPnlTft3TpUqlfv75YWVlJ9erVZdasWbq8PbJkyRJp2LChWFtbi6urq/Ts2VNu376t8d49c+nSJRkwYIDUrFlTbGxspHbt2jJx4kR59OiRxjJnzJghXl5e6mV27NhR/fofP34s4eHhUq1aNbGyspLKlStL9+7dCyzjeX/PYari4+ML/F4Lu0VFRSkdVU2XvKGhocVebl5entStW1frss3NzQv8X9RG1/f52f8p0nT69Gmd3j8lbi+zrRUmNDRU8dei7ebs7Cy//vqrXl7vi+j62anzId9z585h7NixmDhxIr799ltkZWUhODhY42LdI0aMQExMDAYPHoy0tDT06NEDAwcORGpqqromLi4OM2bMwNChQ5GUlASVSoVx48YVWF9WVhYiIiKQlpaGuLg4nDp1CoGBgep57pKTk+Ho6IhBgwYhMzMTmZmZaNKkSYHlhISE4OrVq9i1a5fG/WvWrIGvry9ee+01AE9PHB42bBi6d++O1NRUDBs2DJGRkVq/i42JicGQIUPg7++PDRs2YOHChXB0dMT9+/cLrb9+/ToqVqyI+fPnY8uWLRg7dizi4+M1Rqd98803mDFjBkaPHo2tW7di4cKFeO211/DgwQMATy+mvnr1akybNg3ff/894uLi4OjoqNgcgGQYFhYWmDZtmta6sjAJ+YIFC7BgwQKlY+idv78/RMSot1fl9KPq1atj586daNy4sdJR/o8uXTc0NFQsLCzkxIkT6vuSk5MFgPzvf/8TEZGTJ0+KmZlZgT3S/v37i5+fn4g8/Yvb3d1dhg8frlHTqVOnAnuof5eXlycXLlwQALJr1y71/ZUqVSp0TwV/20MVEWncuLEMGTJE/XNOTo6UL19e5syZIyIid+7cETs7O4mOjtZYTmRkpLi5uUleXl6huW7duiUqlUpGjRpV6OMiBfdQn5ebmyurV68Wa2tr9R7mRx99JO+++26RzwkKCpLRo0cX+XhRuIdqGLrkLcleg5+fn07ryMzM1HmZpraH6u/vL/7+/kZZlz7ouodaml7T80x5D/Xtt9+Wa9euGe290Pseao0aNVC7dm31z/Xr1wcAXLhwAQDwww8/wNzcHD169EBeXp761q5dOxw+fBhPnjzB+fPnceXKFXTr1k1j2c//DADp6elo2bIlHB0dUa5cOXh4eAAATpw4oWtktT59+mDdunXqi4qnp6fj3r17CA4OBvB0ENODBw/Qu3dvjeyBgYG4evWq+jU+LzMzE9nZ2cX6MlxEEBcXh/r160OlUsHS0hLvv/8+Hj16pP4i3sfHB5s3b0ZUVBT27dtXYM/Tx8cHCQkJmD17Nn777bcXTum1ZMkS+Pn5wc/PD9euXdM5J5mOGTNm6FRX1ichJ3J3d8fKlSuxdetWk7xUpc4NtUKFCho/W1lZAXg6Qgt4eijzyZMncHR0hKWlpfoWFhaGvLw8XL58GVeuXAHw9Hyhv3v+5/3796Nbt27w8PDAypUrkZmZiZ9++kljfcUREhKC69evY8eOHQCAxMREtGjRAtWrV1dnB4AGDRpoZG/bti0A4Pz584Uu99mEt5UrV9Y5S1xcHMLDw9GjRw9s3LgR+/btw9dff63x2gYOHIgZM2ZgzZo1aNasGdzc3BAZGalurJMmTcJHH32EBQsW4PXXX0e1atXw+eefF7q+wYMH48CBAzhw4ECB95lKhw4dOiAwMFBrXUZGBrZt22aERETG5eXlha+++gqnTp1Cv379lI5TJL2N8q1YsSLKlSuHH3/8EebmBfu0q6ureg/x+T2l539OTk6Gi4sLEhMT1SO3zp49+9LZatWqBT8/PyQmJqJ169ZISUnR+Ku/YsWKAIDU1FS4ubkVeL63t3ehy61UqRKAp9ea1PWvpbVr16J3796YPn26+r5jx45p1Jibm2PUqFEYNWoUzp8/j9WrV2PixImoWrUqhg4dChsbG0ydOhVTp07FyZMnsWjRIowcORLe3t545513dMpBpUtsbCyaNWumtS4iIgIdOnQwjRGPRCVQt25dvP322+jTpw9atmypdByd6K2hPhswdOfOHXTo0KHQmmrVqsHd3R0bN25Ex44d1fdv2rRJoy47OxuWlpYaHwqrV68usDwrKyud91hDQkIwffp0BAYGIjs7W+M81RYtWkClUuHSpUsICgrSaXl/f96KFSswd+5cnZ6TnZ1dYEaRwl7bM9WqVcOnn36K+Pj4Ao0XAGrXro25c+fi66+/xrFjx9hQy6imTZuie/fu2LBhwwvrDh48iKSkJJ6HTSbN0tISNjY2UKlUcHZ2hru7Ozw8PFCvXj00aNAATZs2LXTnxtTpraF6e3tj6NChCAkJwbhx4+Dn54ecnBwcPXoUJ06cwLJly2BhYYGxY8di7NixcHFxQatWrbBp0yb8/vvvAKDes+3QoQPi4uIwcuRIdO3aFXv37sWqVasKrLNu3bpIS0vDO++8A3t7e3h7exd5YfHg4GD1utu0aaNxmLZChQqIjo7GJ598grNnz6JNmzbIz8/HiRMnsHPnTiQnJxe6zAoVKiAyMhITJ07E48eP0blzZzx69AhpaWmIiopC1apVCzynQ4cO+OKLL9CsWTN4eXlh9erV+PPPPzVqhgwZgooVK6J58+ZwdHTEzp07cfLkScyaNQsA0KNHD/j6+uKNN96ASqVCUlIS8vLy0KZNGx1+U1RaTZ8+HZs2bUJ+fv4L6yIjI/Huu+/qNGG5qdB18mgyPTt37kRAQIDSMUyDLiOXChup+myUW0pKivq+/Px8+eyzz9Tncjo7O0ubNm1kxYoVGjWTJk0SZ2dnsbe3l759+8qCBQsEgNy6dUtdN2vWLPHw8BBbW1tp166dnDhxosDo3QMHDkizZs3E1tZWY0Ti83XPtGrVSgDIokWLCn2dK1eulCZNmoiNjY1UqFBBmjZtKvPmzdP6/ixatEjq1asnVlZW4ubmJr1795Y7d+4U+t7du3dPwsLCxMnJSZycnGTQoEHqc2J///13EXk6ArNly5bi5OQkKpVKGjVqJMuWLVMvY/bs2eLr6yvly5cXe3t7adq0qWzYsEFrTo7yNQxd8hr73MClS5e+cDmmNsq3tOEo31drG9H1s1OnhmpogwYNkurVqysdo8xjQzUMYzbUM2fOiJWVldb1eXh4SHZ2dpHLMbWGOmfOHPVpbKUBGyobamGMfi3fI0eOIDIyEunp6di6dStGjhxZ4MIGRFQ4T09PDB06VGvdhQsX1KPHS4PU1FSNC8AQlUZGb6h2dnbYs2cP+vbti65duyItLQ2zZs1CeHi4saMQlUoTJ06Evb291rrY2FjcvXvXCImICFCgodasWRM7d+7ErVu38PjxY5w8eRJjxozhMH8iHbm6umLUqFFa627cuKHz6HMiKrlSPX0b0atqzJgx6vOgX+Szzz7jFbKIjIQNlagUKl++vE6XGrx//77GRURMlUqlgkqlUjoGUYmwoRKVUh999JH6GtcvsmjRohJdacwY0tPTkZ6ernQMohJhQyUqpWxsbDB58mStdY8ePUJ0dLThAxG94thQiUqxgQMHok6dOlrrVq5cWeilK03FtGnTdJr7lciUsaESlWK6TkL+5MkTTJo0yQiJXs4PP/yAH374QekYRCXChkpUyvXu3RtNmjTRWpecnIx9+/YZIRHRq4kNlaiUMzMz4yTkRCaADZWoDOjYsaNOM37s2LED27dvN3wgolcQGypRGREbG6tTnSnupVaqVEmnC1UQmTK9zYdKRMpq3rw5unXrhk2bNr2w7sCBA1i3bp2RUunG1PIQvQzuoRKVITNmzIC5ufb/1pzZhUj/2FCJypAGDRrg/fffVzpGsU2YMMEkD0UTFQcbKlEZM2XKFFhZWSkdo1gyMzORmZmpdAyiEmFDJSpjatasicGDBysdg+iVw4ZKVAZNmjQJdnZ2SscgeqWwoRKVQW5ubhg5cqTSMYheKWyoRGXU2LFjUbFiRaVj6MTDw0OnqeiITBkbKlEZ5ejoiPHjxysdQyerVq3CqlWrlI5BVCJsqERl2IgRI1ClShWlYxC9EthQicowlUql0yTkShs5ciS/86VSjw2VqIwbNGgQateurXSMFzp8+DAOHz6sdAyiEuG1fMmk+Pj4ICoqSmudLjOr0FPlypXD1KlT8d577ykdhahMY0Mlk+Lj4wMfHx+lYxSLiCgdQauQkBCEhIQoHaPMqFGjRqn4vZdE9+7dUaNGDa11utS8KthQiYiogO7du6N79+5KxyhV2FCJSHF16tRROgJRibGhEpHilixZonQEohLjKF8iIiI9YEMlIsUNHjyYM+RQqcdDvkSkuBMnTigdgajEuIdKRESkB2yoREREesCGSkREpAf8DpWIFFfaro5FVBg2VCJSXFxcnNIRiEqMh3yJiIj0gA2ViBTXr18/9OvXT+kYRCXCQ75EpLgLFy4oHYGoxLiHSkREpAdsqERERHrAhkpERKQH/A6ViBTXokULpSMQlRgbKhEpLjY2VukIRCXGhvoKOX78OAICAjTuCw4OxvDhw/Hw4UN07ty5wHPCwsIQFhaG69evo1evXgUeHzZsGPr06YPz58+jf//+BR4PDw9H165dcfz4cQwZMqTA45MmTUL79u1x+PBhjBw5ssDjM2bMQMuWLbF3715EREQUeDwuLg4+Pj7Yvn07YmJiCjy+ePFieHt7IyUlBfPmzSvw+MqVK1GtWjUkJiZi4cKFBR5PSkqCs7MzEhISkJCQUODxzZs3w9bWFgsWLMCaNWsKPJ6RkQEAmDt3LlJTUzUeU6lUSE9PBwBMmzYNP/zwg8bjlSpVwrp16wAAEyZMQGZmpsbjHh4eWLVqFQBg5MiROHz4sMbjderUUU/cPXjw4AIzuvj4+KgvqNCvX78CI21btGihbnQ9e/bEjRs3NB5v164dIiMjAQCdOnVCdna2xuNdunTBmDFjAKDAdgdw2+O2V3q3vaLwO1QiUtzRo0dx9OhRpWMQlYiZiIjSIcg4/Pz8cODAAaVjEBXwbC/i2V4VkSnR9bOTe6hERER6wIZKRESkB2yoREREesBRvkSkuHbt2ikdgajE2FCJSHHPToEgKs14yJeIiEgP2FCJSHGdOnVCp06dlI5BVCI85EtEinv+SjdEpRH3UImIiPSADZWIiEgP2FCJiIj0gN+hEpHiunTponQEohLjHioZzJkzZ2BmZmbyt8Kmd9JVWFhYsdfn6uqK+/fv6++Nfgm65AwLCzNanjFjxqin2yIqrbiHSmRk165dw/z58zF58mSlo7zyMjIy0LZtW6VjGF18fLzWP5gM9QeVhYUFrK2tYWNjAwcHB7i7u8PNzQ1eXl7w9vaGjY2NQdZrDGyoRAqYN28ehg8fDmdnZ6WjmARO32Z6VqxYYfR1WlhYoHbt2mjdujUCAgLQqVMnVKxY0eg5XhYP+RIp4O7du4iNjVU6BpFJefLkCf744w8sW7YM/fr1g7u7O7p27YqNGzeiNEzdzYZKpJAFCxbgwoULSscgMlm5ublITU1F9+7d0aBBAyQlJSkd6YXYUIkUkpOTg+joaKVjEJUK//vf/9C7d2+88847OHfunNJxCsWGSqSghIQEHD9+XOkYRKXG1q1b4evri++//17pKAWwoRIp6MmTJ5g0aZLSMRQXHByM4OBgpWNQKXH9+nV07twZ69atUzqKBo7yJcX5+/u/0qM7161bh4MHD8LX11fpKIoZPny40hGolMnLy0NISAg2bdpkMjMVcQ+VSGEigoiICKVjKOrhw4d4+PCh0dcbEBAAETH4LT4+Xqc8UVFRRsmjr3NMPT09i73ux48f48qVKzh27Bh27NiB6dOno2PHjrC3ty/2+vPy8vD+++/j1KlTenk9JcWGSmQCtm3bhp07dyodQzGdO3dG586dlY5BRmBpaQk3NzfUq1cPbdu2RUREBLZs2YKLFy9i5syZcHd3L9bybt26hQEDBhgobfGwoRKZiAkTJigdgUgx5cuXx/jx43Hq1CkMHDiwWM/dvXs3Vq9ebaBkumNDJTKCoKAgrTU///wzNmzYYIQ0RKZLpVJh+fLlWLx4MczNdW9RU6ZMQX5+vgGTaceGSmQE06dPh5mZmda6SZMmKf6hQGQKBg8ejJiYGJ3rT548iZSUFAMm0o4NlcgIXn/9dbz33nta644ePYpVq1YZIRGR6ZswYQLeffddnetXrlxpwDTasaESGcnUqVNhaWmptS4qKgqPHz82QiLTERYWZtTp4qj0mDt3LsqV0+0Mzy1btiA7O9vAiYrGhkpkJF5eXvjwww+11p05cwaLFy82QiLTwYZKRalZsyb69u2rU+2DBw/w888/GzhR0dhQiYwoMjIStra2WutiYmIUn4TcmK5fv47r168rHYNM1JAhQ3SuZUMlekVUrlwZI0aM0FqXlZWFuLg4IyQyDb169UKvXr2UjkEm6s0334SdnZ1Otb/99puB0xSNDZXIyMaPH48KFSporZszZw5u3LhhhEREps3S0hKtWrXSqfbs2bMGTlM0NlQiI3NycsK4ceO01t29exczZ840QiIi0+ft7a1T3fnz5w2cpGhsqEQK+OSTT3S6xNpXX32FixcvGiERkWmrVKmSTnV37941cJKisaESKcDW1haRkZFa63JycjBlyhQjJCIybbo2VCUmWXiGDZVIIf/85z9Rq1YtrXXx8fE4efKkERIpZ9iwYRg2bJjSMciEWVlZ6VT35MkTAycpGhsqkUIsLS0xdepUrXV5eXk67c2WZn369EGfPn2UjkEmTNcBerqclmYobKhECnrvvffQuHFjrXVr1qzBoUOHjJBIGefPn1d0MAmZPjZUInohc3NzTJ8+XWtdWZ+EvH///ujfv7/SMciE6To4r7jzqeoTGyqRwrp06YKWLVtqrduyZQt27dplhEREpmfPnj061Xl6eho4SdHYUIlMQGxsrE51nIScXkV//fUXLly4oFNtnTp1DJymaGyoRCagTZs26NSpk9a6zMxMbNq0yQiJiEzH+vXrda5t2rSpAZO8GBsqKW7Xrl0wMzMz6s0UZzaZMWOGTpOQT5w4kZOQ0yvj0aNH+Oyzz3Sub9GihQHTvBgbKpGJ8PHx0enUkSNHjuA///mPERIZT3h4OMLDw5WOQSZo0aJFuHz5sk61TZo0gYeHh4ETFY0NlciETJs2TafJlCdPnozc3FwjJDKOrl27omvXrkrHIBNz8OBBfPrppzrX9+7d24BptGNDJTIhr732GgYOHKi17vTp01iyZIkREhnH8ePHcfz4caVjkAm5cOECevTogZycHJ3qVSoVBg0aZOBUL8aGSmRioqKioFKptNbFxMTgwYMHRkhkeEOGDCnWJNJUtu3cuRO+vr7FutjHgAED4OLiYsBU2pWqhlqjRg2MGTNG/fOaNWuQkJBQoC4gIMBkJitOSEiAmZkZ7t+/r3QUKiWqVKmCjz/+WGvdlStX8PnnnxshEZFxXLhwAaNGjUKHDh2QlZWl8/OcnJxMYhIJ7V/WmJDk5GSNGQfWrFmD69evFxixuWDBAlhaWho5HZH+fPrpp1iyZAnu3LnzwrrZs2dj2LBhcHJyMlIyIv26fPkydu/ejc2bN+Pbb799qbEB8+fPh7OzswHSFU+paqhvvPGGTnX169c3cBIiw6pYsSLGjh2LSZMmvbDuzp07mDlzJmbNmmWkZET/p7AdGm3y8vJw8+ZN3Lx5E1euXMHZs2dLlGHQoEGmcxqclFBoaKj4+vpKcnKyeHt7i7W1tbRq1UqOHj2qUffgwQMZMWKEuLm5ibW1tfj5h/Z1fQAAHWlJREFU+cnWrVs1av773/9K69atxcHBQRwcHOT111+XNWvWqB/39PSU8PBw9XoBaNyioqJERMTf31969uwpIiI7duwQAHLkyBGNdd28eVMsLS1l2bJlGutv06aNqFQqqVixonz44Ydy9+5dre/Brl27JCAgQOzs7KR8+fLi7+8vv/zyi4iIxMfHCwC5d++eun78+PHSsGFDsbOzk6pVq0rfvn3l8uXLGsvcuHGjNGnSRGxtbaVChQrStGlTycjIUD++bNkyqV+/vtjY2EilSpWkTZs2BV7j83x9fbW+Fn06ffp0gd9RYTd/f3+j5tKnwrbDwm4v4/79++Lm5qZ12SqVSi5evFisZeuSOTQ09KVyvwx/f/9SvR1o8+xzQNvt2WeYKdAlr9K3rl27yqNHjwz+Xuj62amX71DPnj2L0aNHIzIyEv/5z39w584ddOzYUWN01j//+U/Ex8dj4sSJSE5ORrVq1RAUFKS+PuPdu3fRpUsX1KpVC+vWrUNSUhL69++P27dvF7rOyMhItG3bFm+88QYyMzORmZmJDz/8sECdv78/KleujDVr1mjcn5ycDADo0aMHAODHH39Eu3bt4O7ujqSkJMTFxWHz5s0YMGDAC197RkYG2rVrB0tLS6xYsQKJiYl46623Xngh56ysLERERCAtLQ1xcXE4deoUAgMD1fP4/fXXX+jVqxcCAwORkpKC1atXo0uXLrh58yYAYPfu3Rg6dCj69euH9PR0/Pvf/0bLli21Hh6k0sXOzk7rHioAZGdn6zQNnCmbNGmSTq+V6Jk+ffogKSlJ53lSjaKknfvZX+g//vij+r4zZ86IhYWFLFy4UEREjh07JmZmZpKQkKCuefLkiTRo0EDefvttERHZv3+/AHjhHuHf91BFRHr27FnoX7V/30MVEfnXv/4l3t7eGjVvv/22BAUFqX9u3bq1BAQEaNT88MMPAkB+//33IjM1b95cfH19JT8/v9DHC9tD/bu8vDy5cOGCAJBdu3aJiMjatWulYsWKRa5zzpw50qRJkyIf/7vFixeLr6+v+Pr6SvXq1XV6jr5wD7Vke6giIo8ePZKaNWtqXX65cuXk5MmTOi9Xl8zG3EMt67iHqr+blZWVzJ0716jvhVH3UF1dXTVmy/D09ISvry/27dsHANi/fz9EROOkW3Nzc/Tu3Vu9h+rl5QV7e3v07dsXGzduLHLP9GX06dMHx48fx6+//grg6XH/HTt2qK9K8/DhQ2RmZiI4OBh5eXnqW+vWrWFpaYmDBw8WutwHDx7g559/RmhoqE6XjHsmPT0dLVu2hKOjI8qVK6e+sseJEycAAI0aNcKdO3cQGhqKbdu2FTg1wsfHB4cOHcKoUaOwe/duPH78uMh1DR48GAcOHMCBAwcUH1JOxWdlZaXT6MW8vDxMnjzZCIkM4/Dhwzh8+LDSMcjEtWvXDr/++qvJXlVLbw21sPueXS7q8uXLsLe3LzDxq5ubGx4+fIhHjx7ByckJ27ZtQ25uLoKDg+Hi4oKgoCCcOnWqxPlatGiB6tWrIzExEQCwbt06lCtXDt27dwcA3Lp1C0+ePMHw4cNhaWmpvllbWyM3N7fIc6Fu3boFEUHlypV1zrJ//35069YNHh4eWLlyJTIzM/HTTz8BgPoQube3NzZu3IhTp06hc+fOcHZ2Rt++fXHt2jUAQPv27REfH4/du3cjICAAzs7OGD58eJk5J5E0vf/++2jYsKHWuu+++67UNqWRI0di5MiRSscgE2Rubo7OnTtj165d2L59O+rWrat0pCLppaEWdr5QVlaWutFUrlwZ9+/fx8OHDzVqrl69CltbW1hbWwN42vi2bNmC27dvY/369Thx4gT69u1b4nxmZmYIDg5WN9TExER06tQJDg4OAIAKFSrAzMwMU6ZMwf79+wvcirpyjZOTE8zNzXW+ziTw9LtbFxcXJCYmolu3bmjevHmhE+IGBQXhv//9L27cuIHly5dj+/btGDFihPrx0NBQHDx4EFevXsWcOXOQkJCAadOmFedtoVLC3NwcMTExWutEBBMnTjRCIiLDsrGxQdu2bTF//nycP38eaWlpaNOmjdKxtNLLaTNZWVnYu3ev+rDvuXPn8Msvv6gH9Lz55pswMzNDUlISPvjgAwBP//MnJSWhdevWBZanUqnQtWtXHDly5IXzRFpZWel8WaqQkBDMnTsXqamp2LVrF7799lv1Y3Z2dmjevDmOHz9erMNmdnZ2aNasGb755ht8/PHHOh32zc7OhqWlpUbt6tWri6x3dHRE3759sWvXLmRmZhZ43MXFBUOGDMH69etx7NgxnbNT6fKPf/wDzZs3Vx/NKMrmzZvx3//+F2+99ZaRkhEVj5mZGaytrWFjYwN7e3u4ubmhcuXKqFGjBho0aIBGjRrBz89PvaNVmuiloTo7O6N///6YNm0aVCoVJk+eDFdXV/W5QfXq1cN7772Hjz/+GHfv3sVrr72GpUuX4o8//sDChQsBAGlpafj3v/+N7t27o3r16rh48SIWL16MwMDAItdbt25dbNy4ERs2bICHhweqVKmCKlWqFFrr6+uL1157DYMHD4ZKpUKXLl00Hp89ezbatWsHc3Nz9OrVCw4ODjh37hzS0tIwffr0IietnTlzJtq3b49OnTph8ODBsLOzQ2ZmJvz8/AqsAwA6dOiAuLg4jBw5El27dsXevXuxatUqjZrFixcjMzMT77zzDqpUqYKTJ09i7dq16j9GoqKicPPmTfXh3kOHDmHXrl2YOXNmke8VlX6xsbFo27at1roJEyaoxyYQGZKnpyfOnDmjdAzTUdLRT8/OQ123bp3Url1brKyspGXLlgVGxj548EA+/vhjcXV1FSsrK/H19ZUtW7aoH//jjz+kZ8+e4uHhIVZWVlK1alUZMmSI3LhxQ13z/Cjfa9euSffu3cXJyUljhNzzo3yfmThxogCQkJCQQl/LTz/9JB07dhQHBwextbWVevXqyahRo+T27dsvfA8yMjLkrbfeEpVKJY6OjhIQECCHDh0SkcJH+c6aNUs8PDzE1tZW2rVrJydOnBAA8uWXX4qIyN69e6Vz585SuXJlsba2lho1asi4ceMkJydHRERSUlIkMDBQnJ2dxdraWurUqSOxsbFFjjR+hueh6p+hR/k+r2PHjjqtLzU19YXL0WUZPA9Vf8rqKF9PT0+lYxqFrp+demuoZPrYUPXP2A314MGDYmZmpnV9jRs3fuEfWKbWUH/88UeNU+/KGjbU0s2op80QkXE0adJEp4kffvvtN41xAqauZcuWGqfeEZVGbKhEpUxMTEyZm4R879692Lt3r9IxiEqkxA01ISEBBw4c0EcWItJBnTp1dLoY+F9//YVly5YZPpAeREREICIiQukYRCXCPVSiUigqKgo2NjZa66ZNm1bg/G8iMgw2VKJSyMPDAx999JHWusuXL+OLL74wQiIiYkMlKqUmTJiA8uXLa62bNWuWXq+NTUSFY0MlKqUqVaqEMWPGaK27ffs2JyAnMgI2VKJSbNSoUYVOTvG8L774AleuXDFCopcTFxeHuLg4pWMQlQgbKlEpZm9vr9MF8R8+fGjSkyf4+PjAx8dH6RhEJcKGSlTKDR06FJ6enlrrli5dqpfpEA1h+/bt2L59u9IxiEqEDZWolLOyskJ0dLTWutzcXJOdhDwmJkanKeqITBkbKlEZ8MEHH6B+/fpa67799lv89ttvRkhE9OphQyUqA3SdhDw/P5+TkBMZCBsqURnRo0cPNGvWTGtdamqqEdIQvXrYUInKkBkzZigdgeiVpX3KCiIqNQIDA9G+fftSN2J28eLFSkcgKjHuoRKVMbGxsUpHKDZvb294e3srHYOoRNhQicoYPz8/9OzZU+kYxZKSkoKUlBSlYxCVCBsqURkUExMDCwsLpWPobN68eZg3b57SMYhKhA2VqAyqW7cuQkNDlY5B9EphQyUqo6Kjo2Ftba10DKJXBhsqURlVrVo1DB8+XOkYRK8MNlSiMiwiIgIODg5KxyB6JbChEpVhzs7OCA8PVzqGVitXrsTKlSuVjkFUImyoRGXc6NGj4eLionSMF6pWrRqqVaumdAyiEuGVkshgatSoARFROoZBJSQkICEhQekYL+Tg4ICsrCylY7xQYmIiAKBPnz4KJzEMHx8fREVFaa0LCAgwfBgyGDZUIlLcwoULAZTthurj46N0jGLR5Q+AChUqGCFJ6cGGSkREBegyaT1p4neoREREesCGSkREpAdsqERERHrA71CJSHFJSUlKRyAqMTZUIlKcs7Oz0hGISoyHfIlIcaXhfF4ibdhQiUhxbKhUFrChEhER6QEbKhERkR6woRIREekBGyoREZEe8LQZIlLc5s2blY5AVGJsqESkOFtbW6UjEJUYD/kSkeIWLFiABQsWKB2DqETYUIlIcWvWrMGaNWuUjkFUImyoREREesCGSkREpAdsqERERHrAhkpERKQHZiIiSocg43BxcYGnp6fSMcqM69evc9oxMlncPvXn7NmzuHbtmtY6NlSil+Tn54cDBw4oHYOoUNw+jY+HfImIiPSADZWIiEgP2FCJXtLgwYOVjkBUJG6fxsfvUImIiPSAe6hERER6wIZKJuv+/fswMzNDQkKC+r4aNWpgzJgxOi9j3759iI6O1msuPz8/hIWF6XWZxpCVlYXo6GicOXNG4/6MjAyYmZnhyJEjygQrA7itanfmzBmYmZkhNTVVfd/s2bORkZFRoNbMzAxfffWVEdPpBxsqlSrJycn417/+pXP9vn37MGXKFAMmKj2ysrIwZcqUAg21SZMmyMzMhJeXlzLByihuq5oqV66MzMxMtG7dWn1fUQ01MzMTvXv3NmI6/eB8qGQw2dnZUKlUel3mG2+8odflEVC+fHk0b95c6RiK4rZqeNbW1jpvZ6V1e+QeKmkVFhYGPz8/bNiwAXXr1oWNjQ1at26NY8eOadSZmZlh/vz5GDlyJFxcXNCoUSP1Yxs3boSfnx9sbGzg7u6OcePGITc3V+P569atQ506daBSqdCmTRv88ccfBbIUdhht9+7daNu2Lezt7eHo6IiAgAAcOnQICQkJGDFihDqbmZkZAgIC1M87cuQIgoKC4ODgAAcHB/Tu3RtXrlzRWPaRI0fQqlUr2NjYoF69eti0aZNO79mznJ999hk8PDzg5OSEkJAQ3L59W6Pu5s2bGDJkCNzc3GBjY4OWLVvi559/1qi5desWQkJCYGdnhypVqmDWrFkYM2YMatSooa65fPkyBg4ciFq1akGlUqFOnTqYNGkSHj9+DODp4bZnv4+2bduq3w+g4CFff39/BAcHF3hNY8aMQfXq1fFsHGNOTg7GjRuHatWqwdraGq+//jo2b96s0/tjKNxWX35bnTZtGtzd3WFvb4/3338fd+7c0ag7ffo0unfvjvLly8PBwQFdu3bFn3/+qVGzfPlyNGjQACqVCs7OzvD398fRo0cBFDzkW6NGDdy4cQNTpkxRv+Zne6t/P+QbFRUFd3d35Ofna6wrNTUVZmZmGhmWLVuGBg0awNraGp6enpg9e7ZO74HeCJEWoaGh4uzsLDVr1pRVq1bJunXrpGHDhuLh4SHZ2dnqOgDi7u4uwcHBkp6eLmlpaSIikpiYKObm5jJs2DDZunWrLFiwQBwdHSU8PFz93IMHD4qFhYX06tVLNm/eLHPmzJGaNWsKAImPj1fXeXp6ajxv586dUq5cOenQoYMkJSVJenq6TJo0SVJSUiQrK0vCw8MFgGRmZkpmZqYcPXpUREROnjwp5cuXl8DAQElOTpakpCSpV6+e+Pn5SX5+voiIPHz4UKpUqSKNGzeWdevWyapVq6RmzZri4uIioaGhL3zPPD09pVq1ahIUFCRpaWmyePFisbOzk2HDhqlrcnJy5I033pCaNWvKihUrJD09Xbp16yb29vZy+fJldV23bt2kYsWKsnTpUklJSZHAwEDx8PAQT09Pdc1vv/0m4eHhkpycLBkZGbJkyRKpUqWKDB48WL2u1atXCwD5+uuv1e/Hs/cQgPz+++8iIrJgwQJRqVRy//599fLz8/OlevXqGu99UFCQuLi4yIIFC2Tr1q0yaNAgsbCwkEOHDr3wvTEkbqsvt61WqVJF2rRpI5s2bZLFixeLo6Oj9OrVS12Tk5MjNWvWlDp16sh3330nSUlJ0qBBA6lSpYrcuHFDRER27dol5cqVkxkzZsjOnTtl48aN8umnn8qPP/4oIiKnT58WAJKSkiIiIr/88os4OjrKoEGD1K/5zp076t/Pl19+KSIix44dEwCyY8cOjdz9+/cXX19f9c+zZ8+WcuXKSUREhGzbtk1iY2PFyspKvRxjYEMlrUJDQwWA+j+GiMiZM2fEwsJCFi5cqL4PgPj4+Gg899kHcVhYmMb9y5cvFxsbG7l+/bqIiPTu3Vvq1aun/oAQEYmJidH6IdW8eXPx9fXVeN7fffnll1LY3439+vWTOnXqyKNHj9T3nThxQszNzSU1NVVERL7++mspV66cnD9/Xl2zZ88eAaDTh1StWrUkNzdXfd8nn3wibm5u6p+XLVsmlpaWcuLECfV9ubm5UqtWLRkzZoyIiPz+++8CQNasWaOuefjwoVSqVEmjoT4vNzdXVq9eLdbW1urX+GxZO3fu1Kh9vqFmZWWJhYWFfPvtt+qavXv3CgDZv3+/iIhs375dAEhGRobGst566y2ND2Jj47b6ctuqk5OT3Lt3T33fqlWrxMzMTI4dOyYiIgsXLhQLCwv566+/1DXnz58XS0tLmTFjhoiIzJkzR5o0aVLkep5vqCIilSpVkqioqAK1f2+oIiKNGzeWIUOGqH/OycmR8uXLy5w5c0RE5M6dO2JnZyfR0dEay4mMjBQ3NzfJy8t74XugLzzkSzpxdXVFy5Yt1T97enrC19cX+/bt06gLCgrS+PnEiRM4d+4cgoODkZeXp74FBgYiJydHfZhx37596Natm/owJAC8++67L8z04MED/PzzzwgNDdV4ni62b9+OHj16wNzcXJ2pZs2aqFGjhvr6p/v27YOvry88PDzUz2vVqhVcXV11Wkfbtm1Rrtz/DVOoX78+srKy1Idht2/fDl9fX9SsWVOdAXh6yPVZhmf/du3aVb0clUqF9u3ba6xLRBAXF4f69etDpVLB0tIS77//Ph49eoRz584V671xcXFBYGAgEhMT1fclJibCy8sLfn5+6uzu7u5o1aqVxu+1Xbt2il8/ltvqU8XZVjt06AB7e3uN1yMi2L9/v3r5TZo0Qa1atdQ1Hh4eaNWqFfbs2QMA8PHxwaFDhzBq1Cjs3r1bvZ3rQ58+fbBu3Tr1/5H09HTcu3dP/dVEZmYmHjx4gN69exf43V29ehUXLlzQW5YXYUMlnRT2H9PV1RWXL1/WuM/NzU3j5+vXrwMAOnfuDEtLS/WtZs2aAIDz588DAK5cuVJgHdo+DG7dugURQeXKlYv3Yv5/rlmzZmlksrS0xKlTp16YSZdcz1SoUEHjZysrK4iI+oPm+vXr+OmnnwpkiI+P18jg4OAAGxsbjWW5uLho/BwXF4fw8HD06NEDGzduxL59+/D1118DePpdZ3GFhIQgPT0dd+/eRX5+PtauXYs+ffqoH79+/TquXLlSIHt0dLQ6u1K4reqeq6g6lUoFe3t79Xt2+fLlAu8X8PQ9vHnzJgCgffv2iI+Px+7duxEQEABnZ2cMHz4cDx48KNbrLUxISAiuX7+OHTt2AHj6B16LFi1QvXp1AP/3u2vQoIHGe9S2bVsAMNo2yVG+pJOsrKxC72vQoIHGfc//9V2xYkUAwJIlSwod9fjsw8rd3b3AOgpb5985OTnB3Ny8wAelLipWrIgePXrgww8/LPDYsymv3N3dCx1soi1XcTL4+flh4cKFBR6ztrZWZ7h37x5ycnI0murzU0mtXbsWvXv3xvTp09X3PT8Qpzh69OiBYcOGYePGjfD09MSlS5c0GmrFihVRtWpVbNiw4aXXYSjcVnXPVVRddnY27t+/r/4DoHLlyurBRX939epV9fsGAKGhoQgNDcW1a9ewfv16jBo1CuXLl8fMmTN1ylGUWrVqwc/PD4mJiWjdujVSUlIwY8YM9ePPMqSmphba+L29vUu0fl2xoZJOsrKysHfvXvWhtHPnzuGXX37BgAEDXvg8b29vVK1aFWfOnME///nPIuvefPNNbNq0CbGxseoPuvXr179w2XZ2dmjWrBm++eYbfPzxx4UeSrOysgKAAg2pXbt2OHLkCHx9fYs8BPfmm29i9erVuHDhgvpQ2o8//qi3htquXTts27YN1atXL3JP4tkh1k2bNqkPb2VnZ+P777+Hg4ODui47O1vdhJ9ZvXq1xs9/fy+0cXJywttvv43ExER4enqiXr16aNy4sUb2efPmwd7eHnXr1tXh1RoPt9Xib6vff/897t+/rz7su379epiZmam3v2fZT58+rf7D4uLFi9i7d2+hF6NwcXHBkCFDsH79+hf+YWdlZaXzEZSQkBBMnz4dgYGByM7O1jhPtUWLFlCpVLh06VKBQ/nGxIZKOnF2dkb//v0xbdo0qFQqTJ48Ga6urlqvwmJubo558+ahf//+uHv3Ljp16gQrKyucOnUKGzZsQFJSEmxtbTF+/Hg0a9YMwcHBGDRoEI4cOYLly5drzTVz5ky0b98enTp1wuDBg2FnZ4fMzEz4+fmhS5cu6g/7zz//HIGBgShfvjy8vb0RHR2Npk2bIigoCAMHDoSzszMuXryI77//HmFhYQgICMCAAQMQExODoKAgREdHIzs7G5GRkXqbtPmDDz7AokWLEBAQgDFjxqBWrVq4ceMG9u3bB3d3d4waNQoNGzZE165dMWzYMNy7dw/u7u6YP38+bG1tYW7+f9/YdOjQAV988QWaNWsGLy8vrF69usApDdWrV4dKpcKKFSvg6OgIS0tL9QdmYfr06YOBAwfC0dERH3/8scZjHTp0QMeOHdGhQweMHz8eDRo0wN27d3H48GHk5OQgNjZWL+/Ry+C2WvxtVaVSISgoCGPHjsXly5cxduxY9OjRA/Xr1wfw9HSkWbNmoVOnTpg6dSosLCwQHR0NZ2dnDBkyBMDT01tu3rypPtx76NAh7Nq164V7p3Xr1kVaWhreeecd2Nvbw9vbW+MPxb8LDg7G2LFjMXbsWLRp00bj8HmFChUQHR2NTz75BGfPnkWbNm2Qn5+PEydOYOfOnUhOTtbpfSgxowx9olItNDRUfH19Zd26dVK7dm2xsrKSli1bqkeFPoPnRub93ebNm6V169Zia2srDg4O8vrrr8vEiRM1RsGuWbNGvLy8xNraWlq1aiX79u3TOnJSRCQjI0PeeustUalU4ujoKAEBAepTN/Lz82Xs2LFSuXJlMTMzE39/f/Xz/ve//0nPnj3FyclJbGxsxMvLSwYPHqwxUvLXX3+VFi1aiJWVldSpU0eSk5PF19dXp5GTz+eMj48XABqjKW/fvi3/+te/xMPDQywtLaVq1arSo0cP2bNnj7rmxo0bEhwcLLa2tuLq6ipTpkyRDz/8UF5//XV1zb179yQsLEycnJzEyclJBg0aJCkpKRqjd0Wejt6sXbu2WFpaqkeUPj/K95m7d++KSqUSAPLHH38UeI05OTkyefJk8fLyEktLS3Fzc5OOHTuqR54qgdvqy22ro0ePlqioKHF1dRVbW1sJCQmRW7duadT99ddf8o9//EPs7e3Fzs5OgoKCNEaoPzuly9nZWaytraVOnToSGxurHtVc2CjfAwcOSLNmzcTW1lZjBHpRv59WrVoJAFm0aFGhr2XlypXSpEkTsbGxkQoVKkjTpk1l3rx5L3z9+sTZZkirsLAwHDlyRPHRm/RUXl4eGjZsiGbNmmHFihVKxzEp3FaLr0aNGujVqxfmzp2rdJRSj4d8iUzc2rVrcenSJTRq1Ah3797F0qVLcfLkSXzzzTdKRyOiv2FDJTJxdnZ2iI+Px59//oknT56gUaNGSElJQdOmTZWORkR/w0O+REREesALOxAREekBGyoREZEesKESERHpARsqERGRHrChEhER6QEbKhERkR78P9h77lq8++6BAAAAAElFTkSuQmCC\n","text/plain":["<IPython.core.display.Image object>"]},"execution_count":40,"metadata":{},"output_type":"execute_result"}],"source":["#提示：混淆矩阵\n","Image('Snipaste_2020-01-05_16-38-26.png')"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"oRZVVrUIcQfo","outputId":"ccd19f75-d862-4c97-b39f-d1d2a38c315e"},"outputs":[{"data":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAr4AAAE6CAYAAADwTlEmAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAEXRFWHRTb2Z0d2FyZQBTbmlwYXN0ZV0Xzt0AACAASURBVHic7N0xbBvZ3vf3L3MvEgYBAm4TcqvlLYLVRfDCvMULjStzK/NWnFsEGlXmIoV5K81WojohKcgFXkC8lWngeeHZAE84rjQCkpibFBo/b8FxJTopzAcpxKcINK40SCN2JwVJiZRISZYlyzJ/nwvhipzhnP8cbvHz0ZlzUsYYg4iIiIjIN+4/u+8CRERERES+BAVfEREREVkKCr4iIiIishQUfEVERERkKSj4ioiIiMhSUPAVERERkaWg4CsiIiIiS0HBV0RERESWgoKviIiIiCwFBV8RERERWQoKviIiIiKyFBR8RURERGQpKPiKiIiIyFJQ8BURERGRpaDgKyIiIiJLQcFXRERERJaCgq+IiIiILAUFXxERERFZCgq+IiIiIrIUFHxFREREZCko+IqIiIjIUlDwFREREZGloOArIiIiIktBwVdEREREloKCr4iIiIgsBQVfEREREVkKDz74Ro0UqdT8n0Z03avE+HaKVMrGjz/lM2fnT+q4fps3FDVI3XkjIiIiIt+eBx58YwaXZMCtxw2uFxEHDPYALPK565wf0Uh9z/re2flWzWCMoWZdq8EbiX2b1OMtyvn83TUiIiIi8o164MF3EljrdM0oeI5+jmiXASIGkxHc2MeeHhGeHjWNB6OAXM6TH70xHgGeM3ocNUilHrMFwBaPUw0iIhqpFKnUdNCevDf+sX2mB5NHI8Q2fjzd1qKgPjrn+/U9APbWv8ceDzXHvv1po9xRY3Zk/Fxd5/vJPjcEPtve7Aj55NhZDef6ZXLtRnR2nUZ0eX/P9NfFfjrrx4vnamBcREREZpiH7KhtymAot83RvPcpm/bR9OvZn3J7/KluffRevWuMOTLt8sVzoW66xpijdnn2/XLbHE2uX++ea5+L545OGLdRNuXzbU2uMaNr6ueuVe8a063Pq3N0bK7JfV7VDwuuN7+9Ub9M39Pkche+n/H1y+VJH5ZN++jy/l74fYyvOfk+Tu950X8TIiIisvQedPC9EELP/5wLsqfhaCbozglPs61cCHSTAHj+eqMAORXUztLYuWtMBdmzVHlJ8DULQ+RM0LzqGuctCPxz/0Fw/h8TF/phfE/TgXNBP8/UfFV/n97n/IA9ueak5gvfjYiIiMjYg57qMBjNc5ir3jWYmsXZdAjYejz+U/njrakzY0J/DyhzOnV25s/937M+M/93Mq/47PwoHF3PyueYmX5xOuE3R94C2GMw4GxqxdQ58WWTlUc3y6iMPLmp88vtFs5kXnI+z2iGx4BFz+jNTBkY90M5n4c4ZNQNbVrO6cTl0dSRmkUc+uyda29mXvPknsb1Td9TvWiNb2HvYs1waX9P+rberWGN+9IJDMYEODnI5acnVUeMTq9TvMO51iIiIvIwPeDgexZA20fjub3d+uhQuU1lEnxO5++2OZqZB3w+GI+CVuzbpL5fh/bR7DXrxXHwOv8g3LkgfGG+8JxzJiH29JpnobC+ILGdBt1x2p6cb00/jXcuHM8azbd9vDWZDz2ZB13GKeau+OyC9qbrGwfj6fpHn5n8A+GsD5zi2TUu7++L/8i4YBz29wYDYr/BFlBuV1DuFRERkfMecPCdsxKDVaNbB/bW8SYDqLn8KATt+YQxXFi6bCaono3+jsJZjN+YGhVl6vxFQfhCexD71dEoZtmhmLs4EjpvFPnC3Q6m64J8vgzAVnj6mBeN8Qju3PAchaMH8iZ1R97sSPaF0eLJg2mjfpq0F02eFpw8JDd+OO7C6HvUYFTO5PsZ99O4Dyb3fXl/nxsph3MPxXHW31GD6voeUKfmzA/nIiIisuTub5bFZ1r0ENNkTujU+3Mfypo7t3bRg1Znc0Zn5xXXTff8g22L2rvwkNj0PNfJnN/JPNbzZh9uK48mt17xAN35S8x/cO3s/IsP0J22tfDzk3uY02/l8uz3M6efPr2/zz/4dvEa5cWTh0VERGTJPdgR38mf1i/8ad6qjP6EPzXqa9UMk7+gA1Dvjqc5nB99zeG02qORz9G7tNtlpuWKztnxcp78eIrA9Nq6F9obT8cYNTkZ3Z0aqb4winyeRXHqelY+BzmH4Gi61vF9Bc7cqQqno+GTitpt6rMnUDPdmffK7SOC6fm+MzdVpn0UjOfq5nBqMxfnqOWMfnWKo3rm9NO1+tsJOJp5r07X1Kb6aTIqPDqm0V4RERFZJGWMMfddhMjNRTQm6ypP/YNGRERE5Lw/3ncBIjcVNVKcLdAxvYqGiIiIyEUPdqqDLLvp7arLtI9qWslBRERELqWpDiIiIiKyFDTiKyIiIiJLQcFXRERERJaCgq+IiIiILAUFXxERERFZCgq+IiIiIrIUFHxFREREZCko+IqIiIjIUlDwFREREZGloOArIiIiIktBwVdEREREloKCr4iIiIgsBQVfEREREVkKCr4iIiIishQUfEVEROQrFOPbKVKpPFaxSLFYZCWXIpXKsTJ+beVTpFI2//E/dfCqBVKpFKlcCbdZo2oXKRZtqq0Og+HtVxc15tWSIm+NXhdXcqRSKf6n/61H0CiRS6VIWQ2ic7Uk/RC/ZpFKpbBqPmE/uf1i5YwRERER+eocmXb5kdnsnky9xkDZtI/Gb510zeaj8etu3QCGcttMDpvDV6YMhtW6OTi50MBn6dYx5fbhzGvA1LunjZt2efL6cFw7JrvWNofnL3aybzbYMPu3XKNcpBFfERER+TqlHWwrfclxC6eSW3w8b1N9DrzbohVeYyQ16ROGPeJrjRCXcYr5S47nKVWfTwoFsmSz8PH1Oq4/mD01nSZHjvQltyq3Q8FXREREvkI5HL+GdcVZBbeFszD7ZsjkR7/FyTXSbD/gp5+2uU5GtmrBJe2OWy+1qJ3egEUzaLOWhb11h8b5OQ/yRSj4ioiIyDcqIRmMfsvnMvdayagIh5b3nCzv2LJdOprO+8Up+IqIiMg3aRi12H4JrO5QKX4d8wgypSZBfRU+vqRS9Rncd0FL5o/3XYCIiIjIrRkENJsxw55PEEKxvotXtVm577pOpbFqPu2exfrrdZxCnrB23zUtDwVfERER+XbkbVzXIYdL84pTo0aKx1sX31//PsX61Oty+4jgqgm9n1YkTssjDP/Kyy2XZnH7Fq8tl9FUBxEREVlKlnvM8fHUz5tNoMyrD7Pve/Ztht6xTIlmUGeVd2zZlStDutwOjfiKiIjIckpnyExP/c1kxm9nJr/ebfNWDb/dw1p/zce7b07QiK+IiIg8CJPlvxKGX+NKYOMVGuK5xQ0ZJvPrzjtNWmvZOy1Nzij4iojIA5cQVAo0e/ddh9yVYdyj4zVo7gG8pdXw6PTisyjc7+B5wehF5OEFIV9s59+kTxg02PZGL/1GkyDsT3Iww7hH0KhSe/uWWrVBMFX3SA7bC6ivfqF6l1zKGGPuuwgREZEbG3iU/vQzvc19Bo0iX8eiVfIgDTo0gxirUsH6Cpb9ldun4CsiIg9ar1ngL7+8B56xe+xhK7CIyAKa6iAiIg/XMMRvlmi3nwG/0QoG912RiHzFFHxFROTBSjoeXtXGsStsZuH3ZsC8qb5x2KRabRL4DZyijev3z+ZZJn38WpWa5+O5JYqVJlECJAP8ao5UKkUjAoYxPb+GlUqRsn1ixvM3W1VK+Rp+x6WYS5GrdkiGfXzXptIMCBo2VsmlM5gufH6bSd/DyaVIpYrUwnhyMr2WTarUpPc1PtQl8pAYERGRB+nQvHr6yOwcGGPMidnfzBrIms39k5mzjnbXTLbcNofj1wc7jwxkTf3AGHNyYOqrj8xmd/yZ413zDAxru+bIGHPULhvA1LuTq3VNHQzltjkyxpwcH5gXTzCQNWvtA3Ow+8LsvOqaf9l5ZGDTdMd1tssYNvbNibm6zcNXTy/cR7eeNc/fHC/siW4dA9f4ObsRkaWkdXxFRORh6gU0cQkKAGmKTo1Hv/7Cr16HWtFmNNU3wqu+xvZa5McfK1QDuoUhKwWI/W22hhUOrPEjcRmb5od9qmmLHBBfbHVGOpMbr/fq4joFRv+DYa/GzovcaZsADOLRk/7B5W1iuzyv/XXqPiLCVgWnv3jysuUec1y9Rp+lNQFalpuCr4iIPEBDQr/BMO0QNCd7XvXJAe9/axFs21TyQC/C/wjO9G4E6TxWESChE+5B3mF6X67MShHrM6tLFxzcQkzkN2hEMExglMQTele1mSlRrT3i5S+j+3AGAS2nhHvZchXnN2K4hlQq9WkfEBkzD3hdBAVfERF5eIYRgVehNWhQnAp8lfyQ7/72G82gR8UtwHDIe8BKTpPn9EVIYmAQEwO3uintIKBSbLDid2g6Q3x7i5ef0GbBqVH+ZZ1aq0MOn6rduHyZtmFCcp35v1MB+SGHF5Gb0sNtIiLy4CQdD9+1Z0IvQKZUYQN43/AJh0A+Txl46YfM7GcwCGgGkLey8N4jnHkibkjUbBHduLoY3/0bvxVcnAuLweau12bOxt3M8vHXCpXQpXTFEHTU/I7vvrvGT/PmdyXyLVDwFRGRB2ZA0PqNSnFOGkxb2BvAx1/xOskoQG5k4bcqTqNDP06Iez5Vt0+xlMNyGjzlPb9Uq3jRgCQZEDYcWnkbC8jkVsgCQRgxZEjcCQgAkoTFG4PFDCJg0GOQAIOQIAKSIclweGWb4xuhWG3wlI/knCKFK3rEqhmMucZP7XMncYg8cPf0UJ2IiMinOzky+/WnJgumvPPGHBxNr+BwYo4O3pid8ngFg+xTU98/Mifm0OxuPDU/ZjHwg1l99sIcTC2QcHzwwjxb/cEAJvvjU7Oxezh1zUPzZuOp+WFy7E3XtJ+tmmf1tnnz4cgc7r8wa1kMrJrN9oGZlHP4ZsM8yWLI/mjWdrpm/8UTkyVrnmy8MUdXtjlxYHZ+fGJezTskIjeindtERES+RgOPUhVancrs6hC3Jsa3v2d97wdWn+RJA3H/Lf/6McuPT1bIAcPBW979W5l/+pcqf/jnGj+/fA/Zp2zUCgzDiH6SYcWpUquUyN/yXtFRI8XjrfO1wA+rT0ZtxX3e/utH/sf/9YB/93/XqG79zsfVOt2whjVVS9IP6Xg11n99x+pmm0alRHHltla3GBL3Qjqt6b4pnX5fcb+D//J3Cv/0L1T/8M/Ufn7Je7I83ahRGIZE/YTMikO1VqF0yx04jHuEndZMm6Wzwuj4L/n9v/sP/C///f/DP35+ybvsGu3Ix8mfu0bQoPL317D2Aq9mUyzkHva24PedvEVERGTi2Hw4+GCOTw5Ne+2HS9fu/XxHpl2eWk/YHI3WG6Zs2kfjt066ZvPR+HW3PhpJH69hbIwx5vCVKYNhtW4OTi408Fm6dUy5fTjzmpk1lUfrI49ej9dKBpNdO1uz+dTJvtlgw+zfco1TxV3sm7Hj3Wfj+xivAT3dv+bQvCqP/mJQv+0OHBU2p83Twsyz8frW3c3xX0lW66Z7oYxD8+rJt/OXB83xFRER+Uokgcuf//Jnvvsv/0Qt3aRWuuN1d9MOtnXJ+F3awqlcst5F3qb6HHi3RStcPOv5VNInDHvE19qBroxTzF9yPE+p+nxSKJAlm4WPr9dx/cHsqek0OXKkrztU+Ul1Xi5j2eSSRX2Txx51IFut8JJ546eF0Q9DerdTGHZuPFc9A9lsFt5tUd2OmL16mnQmc/2++8op+IqIiHwlMnaTw+4++wdH9D37jqY4TORw/NqVaxYX3BbOwuybIZMf/RZfZz21fsBPP21znYxs1YJL2h23Xmpx9ryeRTNos5aFvXWHRvQZ4fAT6rxSzqZVXfx4YuasA7m64j7BTz+xfTuFYbeqpw9Oul5AfRXe/2pTOf8Ph2+Igq+IiMhXI0PeKj6geZQJyWD0Wz73FewKl3doec/J8o4t26VzG/nwc0QNbP/y/f+Ssw68sNL03Ylo2P7szoQZi1prh1U+8nrdodlb9NmHTcFXREREbmQYtdh+CazuUDm/qPI9yZSaBPVV+PiSStVncG+VJISBf/kpw4jWqAPZqRS/2D92kjBgbmUFF7+9RpZ3/FJt8DmD5l8r7dwmIiIi1zcIaDZjhj2fIIRifRevarNy33WdSmPVfNo9i/XX6ziFPGHtCzU97pscjFdOeE/6wiyHAUGzSTzs4Y86kF2vin2nHThuc1QY/Y7P+4uFAZB3PILBgMdbW9iVPJFfvMvCvjgFXxEREbm+vI3rOuRwaV5x6mhJsovvr3+fYn3qdbl9RHDVhN5PKxKn5RGGf+XllkuzuP1l6jztm5FqKY1zYaZDHtt1cXLgXt2BpOYXRmq2MI4C55Jtt8/aHBdG+mJhY2msWoud4C/88tqlZgfYV5T5kCj4ioiIyJ2w3GOOq1NvRA2++2ufVx887OmUlr6D2a2ZEs2gTu/xFlt2hSwuxS9cZ7rYwBt+xgQGy+V4tjAa3/2V/qsPeLOFfdr84HSRhje8ZGpFAddvE1nrvF63CbPWlf/IeSgUfEVERORupDNkptNVJjN+OzP59W6bt2r47R7W+ms+XnriXdWZnr3uTT4/W9go4KYzZD6zA9NXFZZ38IIBg8dbvLu08x4WPdwmIiIicLqYVsLwa3yoabxCQzy3uCHDZH7deadJay17p6U9bEOG8fx+TVsurfrqPdR0dxR8RUREltww7tHxGjT3AN7Sanh0evFZFO538Lxg9CLy8IKQ/pdaKizpEwYNtr3RS7/RJAj7p5s9DOMeQaNK7e1batUGwVTdIzns8Rq1t29I3OvgeeM1EqKAoDOnb5I+Hc9j1IMRnhcQ3nEHjr5Tb7x6Q0QQdC62mfTpeC61f8A/ai5ep39uE400hZpPe+3rWLHjNqSMMea+ixAREZElMOjQDGKsSgXrK1j2d6Gvts4BnWZAbFWofF2FPRgKviIiIiKyFDTVQURERESWgoKviIiIiCwFBV8RERERWQoKviIiIiKyFBR8RURERGQpKPiKiIiIyFJQ8BURERGRpaDgKyIiIiJLQcFXRERERJaCgq+IiIiILAUFXxERERFZCgq+IiIiIrIUFHxFREREZCko+IqIiIjIUlDwFREREZGloOArIiIiIktBwVdEREREloKCr4iIiIgsBQVfEREREVkKCr4iIiIishQUfEVERERkKSj4ioiIiMhSUPAVERERkaWg4CsiIiIiS0HBV0RERESWgoKviIiIiCwFBV8RERERWQoKviIiIiKyFBR8RURERGQpKPiKiIiIyFJQ8BURERGRpaDgKyIiIiJLQcFXRERERJaCgq+IiIiILAUFXxERERFZCgq+IiIiIrIUFHxFREREZCko+IqIiIjIUlDwFREREZGloOArIiIiIktBwVdERERElsKDD75RI0UqNefH9onvtuFRO43oGifH+HaKVMrGv9Oi7lnsY0/3/Sf10R3VsOj4+OdLliYiIiL364/3XcDniRksCi5763zfyGNq1t00bdUwpnbNk3M4gcG5m0q+HoMBewBWnhwQj7+ccj5/bzWcF3nr7AH1ruGu/tMQERGRr9MDH/EdMNgDqNM1BjP5OWpTBogGxKejrQ2i09G+BhHnR4svjsbGvr3w+OSzkxHD2XPPjTjOG4U8N/I4Oyo6VTMRjZuMYo9HW20/Pq3V9uMr2r1Yl32uUy7c59TnzwfdwejLwcrPi6BnNV7aZ41o5rzz9Ux/h43orIZ68XyqHfXj463Rq63HF6+1qA8u/rcx9Z1cow8Xfrfn/ns83x8L6xMREZGbMQ/ZUduUwVBum6Opt7t1DGCod40xXVMHQ7k8OhdMuX10ds7MT910z19j7vEj0y5Pve7W55w7af/seLl9NPN64flzamaq9ut1TXl0frl8VvukvxZdc0Fdk7Im11x0fNJno9eTPiqbuSUvaOt8LWf1T37Orjf/O1rQ5px7P+3u2cJGfb/ou79hH1723ZbbR1f2vYiIiHy+Bx18FwWxmfAzFVQuhJPTwHwupJ1+5mLIGgWRSXgZfX5ybFEondQ589mZ8yfvXQxXk3NOg+w1g+9ZKDwf1qfC1KQf6t2Z4xf6aUH6mq3p3D8GzvXRNQqeaevsu518B9f/jqb/AXOu4Ln/UJpbx4JzZv9Rdf6aN/tuz845u5dP/b5FRETkag86+C4e8ZsKPnOCzOLAPAoeV4aO84HwwmjddPBaENhmgtW5c+bUfFW4njUn5C4cyRzfx3VC4dR1L4xKnv/8FaF5+p7mjZxevN/ZcDj3O7oitJ4feZ9tfxI6L/bTxT78lGB99Xe7aBRZwVdEROR2PeA5vpMH28q0j6bm9xqDMTUmMzxP5506xdOHnSZzT+vd858LcHJXz0298NCWVcOYI9rlyRlbPD6dFzqZh2yRz7Hg4avZcy7WfHavTnHBfNkZZ3Ofi2cdMZpHWm5zZM7dd8268qGw0bzV71ln8vkudaba+KQH2ybzbSdzsyd9N7m/Ofd7Wr9DccF3NO+7nrmF8fHRZ849GDm+LljUjOHo7Ms8mw98WkOeeXd1s+926nP17rn/Hg2Bc53vW0RERK7jAQffc4Fy0VmDPc4Hxnx+FGq2wnHyObfs1uR4NIhnj48fvpofjEcrNxhj6NYB9hgMuBiW8vnRg3dbIZPcFTUeswVQL2LNrfl693pq0ub4eqPy8qPf93zCGC4ssTapKxqMHzCbPMA1Oh6HPnucBbbYb4xqHt/XJz3YFoUz90vksT5zf+P7PQ2jLAzmp99h7FNdv+wfLDGhP+rXUYln35cxBhM4M9fNOcHo/dGXyd5gMNWHAwZM9+H4IccbfbfzPjfp+wbT2VxEREQ+0xcfY74tn/Sn+XN/mp77p+WpB6LmPmh0fq7p6JpXTrc4/2DbgukCi64/914nrxdNI7jQ5uTtBdMcRkfnToU4P8d40edv48G2y+7vwtSGRddY1OZV0xQuveb5+1rUhzf4bi/5nKY5iIiI3K4HG3xPg9ilj71f8oDVTPidE5ZmQtD08YvXvBgKz4LN7INtp42fCzrTQWhOzQtWhbjew3Tnb2teYDvX9tzgNVtzvT0dTj/9wbbpOsrt9uz5c+5vdP7s9zT3Gp/zYNv0eXPn+F7sh4t9+Inf7YLPKfSKiIjcvpQxxlxnZFi+JjG+/T2DmjZhEBEREbmuB75z2zIahV7fOSJQ6BURERG5No34ioiIiMhSeMCrOoiIiIiIXJ+Cr4iIiIgsBQVfEREREVkKCr4iIiIishQUfEVERERkKSj4ioiIiMhSUPAVERERkaWg4CsiIiIiS0HBV0RERESWgoKviIiIiCwFBV8RERERWQoKviIiIiKyFBR8RURERGQpKPiKiIiIyFJQ8BUREZHlEvvYqRSpvEWxWKRYXCGXSpHKrYxfW+RTKVL2/8z/1QtolFKkUilSVoVa06VSLFIsVmj4PZL7vhf5JAq+IiIisnwebdLtR4RhSOhvYwFY2/hhSBhG9LubPOI/578p2FQq5dFn7CoNt4kXhgTbOfz1v7DiBMT3eR/ySRR8RUREZOmkHRsrfclxy6GSW3w8U3SoPIKPr7fxe7dfn9yNP953ASIiIiJfVM7Br111UgG3VQBYMKKbI5cH3r9nOLzN4uQuacRXRERE5JPFxAOAJ+QuGRmWr4uCr4iIiMgnGvgNGu8h+7yGnb/vauS6NNVBRERE5DpCj2Y6oN8J6Azy2C/22a4Wydx3XXJtCr4iIiIi11Gs4LoWuI37rkRuSFMdRERERGQpKPiKiIiIyFJQ8BUREZHlNlmOLBmilcm+bQq+IiIit2UY0wsalFKjLW6tSo2mWxltg1tp4PfucIPbJKRh2zTC67TRx6sUqXj9u6vnQRgS9zp4jSZ7AG9bNLwOvXg4dTzA86LRy8DD6/SIlY4frJQxxtx3ESIiIt+OGN/+nvU9qHcNNQsgIawV+enXmLXdHr59Bwu/Dnxsy4VmRODkLz93GFFbeYzvdOk3LC7ZwEzkm6IRXxERkTuXoehUeMRHXm/73MkOt3mHII6vDr0AaYvGwDBQ6JUlo+ArIiLyJeRy5AHeax6pyH1R8BUREfkS4pgBwJMc38U9glaVUr6G33Ep5lLkqh0SIImaVKs1apUiK1YVrz8Vk5M+fq1KzfPx3BLFSpMoARiS9Du0qiVWGtHp6YOOS8X16HQ8XLuI24kBGA4i/EYFq+ITn12cqFWl1gwI/AaVUoVG0B+H9IR+4OE6K+RqHoFbIp9KkcrbtHqK8fKAGBEREblFR6ZdxgCm3p28d2jaa1kDWfP8zbE5OT4wL55gIGvW2gfmYPeF2XnVNUf7m+aHZ7vmePyZV08xZDfN/okx5uTA1Fcfmc3uyeiSx7vmGRjWds2ROTHHBy/MEzBMGj3ZNxuUTftoXMLBjnlU7xpjTszx0a55DoZy2xyNa95dy5rsxr45OS35lXkKZrV+MHrvsG3KYFh9bnYPx9fPTtpf1BXjz1z5M1WnyB3Szm0iIiJ3JPSapIM+naDDIG/zYn+bajED5MhkAFxcp8DofwO80q+U3OPxFrh5CqVH8PuvBNE2K/E2W8MKB9Z4Vm7Gpvlhn2raIkcacpnZrXOHQ4bs0fQiSjWLTKGCnwZIk5lMu5jo+Wy//kilWzyb85u3cZ//zF+3WoTVFqXJAbuCnQdYwbLgH6/7DHyY+7hezsY7Pr5WX6W17698AQq+IiIid6RYcbn2Drdxj87vkOQ9mqerjFXY2YFM+iO9cA/yzkzAzKwUsRZdL1Oi1l7DWn/Md60yO76Ha81Pl3E/5D3gzF6AFesJvHxJr9+ilL/GPVyQJpO5ncfnUqnUrVznoTBadOtOKPiKiIh8FUYPveXtCm7pfECN8RvAICZmwejqHHnHp5+3qFYa/PJ4hWCnQ8ctXFzJIT16Z5AkMDVunE5ngCw3X/phSJJcbw5wOpO5tBkFQbkNerhNRETka5DOkAOCTu/cqg99PG/A91YW3nuEM2uhDYmaLSLmiDs0g5iM5eL3++xv5nj7S4t5+1vkCiWeAkFvdkOLOB5AtoK1csN765qZyAAAHCNJREFUigMq333Hd1f+VAjiqy8n8rkUfEVERL4GGQv7eZaP/3CwGx36cULc79AoVUkKFk+cBk95zy/VKl40IEkGhA2HVt5eMN0hIWx1RitJkKFolyCbmQzuzso7bNdX+dj06JwG4x6hH7PWrFK86YhvziEwBnPlT4BzB3t6iJyn4CsiInJbhjG9wONsh1uPTi8+N4I7ZBAGBBFAQOBPtsDNUGpF7G4U6G/9lT9//x2W2yHf7OAWgHwF/+AFz+jw8+M/sWJVCVaaeHYOhjFREIxGfoOAYDCaspAf+tRq/mg5s+0BLzrbFNMJ/SAgAIgCgigG0li1DgeNNK1KlVbQwXNb0OjhO3kgod8ZXz8MCePR8mmjewgJQw3XysOgLYtFRETk6xf72N+vs/fDKk/yaSCm//Zf+Zj9kScrOWDI4O07/q38G++3/2v+99rf2PodWH3GppMhDnoMyFOqulSdAre7iERCPwwJGtNtFk7nYg+iFv7rf8X9Pz5Q+H+b1H5+yXuyPN2oURiGRP2EzIpDtVahlNdeenfqflZRExEREfkER21TfrRpJssYn64RfLoWsTEn3U3zaPz6qF0erRF8tpiyOd7fNI/AZC9be/izSrzY5sTBziOz2TXGmK6pX1i7+NC8KmNg1dQPTi58Vm6PpjqIiIjIg5B2bKxLBkTTlkPlkrnCmaJD5RF8fL2N31t83sQw7hGGfeY8D/jJCsUKw2TRlJA8dvU58I6tVngr7cl8Cr4iIiLy9cs5+LWFqxaPFXBbziXLveXI5QHeM7zGKmtJuM1PPwX0rz71agWXZmlxZZlMfvRLnKBNoO+Ogq+IiIgsiZh4APCE3BdcRSL2bRpz15w7kySD0S/53C3PP5ZpCr4iIiKyFAZ+g8Z7yD6vjbdd/iKt0vH3Lj9lGNHafgmsslMp3ny/ELmSdm4TERGRb1fo0UwH9DsBnUEe+8U+29Xi3Y6qhh7N9HiIdxDRiqBy4aQBQbNJPOzhByEU6+x6VeybbhYi16LgKyIiIt+uYgXXtcBtXHFijG9/z/qFwdk9Hqe2Zt6pdw2XTjeetDlmF2L8CyflsV0XJwdu84rS5NYo+IqIiIiQw/aOOZ56Jw4q/PnnFd4c12Z2x0t/4nBx3glwNX/hq6DgKyIiIgKkM5mZ+bXDNECGTCbzeVMj0hnN2/1K6OE2EREReXgma34lQy3/Jdem4CsiIiIPyJC418FrNNkDeNui4XXoxcOp4wGeN364LPDwOj3iO03HCf0wwPPGE4SDAD+c02bSp+N5BABEeF5A2Nd2FV9Syhhj7rsIERERka9NEnl4UQ7bLZG/72LkVij4ioiIiMhS0FQHEREREVkKCr4iIiIishQUfEVERERkKSj4ioiIiMhSUPAVERERkaWg4CsiIiIiS0HBV0RERESWgoKviIiIiCwFBV8RERERWQoKviIiIiKyFBR8RURERGQpKPiKiIiIyFJQ8BURERGRpaDgKyIiIiJLQcFXRERERJaCgq+IiIiILAUFXxERERFZCgq+IiIiIrIUFHxFREREZCko+IqIiIjIUlDwFREREZGloOArIiIiIktBwVdEREREloKCr4iIiIgsBQVfEREREVkKCr4iIiIishQUfEVEROQbEePbKVKpPFaxSLFYZCWXIpXKsTJ+beVTpFI2//E/dfCqBVKpFKlcCbdZo2oXKRZtqq0Og+F934vcBQVfERER+YY8YrPbJwpDwtBn2wKw2PZDwjAk6nfZfAT/1X9bolJxRh+xKtTcBq0gJPRs4r//lT8VG/QUfr85Cr4iIiLy7Ug72Fb6kuMWTiW3+HjepvoceLdFK0xuvTy5X3+87wJEREREbkcOx69deVbBbVEAGMw7miGTH/0WJxry/dZoxFdERETkVEIyGP2Wz2XutRK5fQq+IiIiImPDqMX2S2B1h0rxkikT8iBpqoOIiIgst0FAsxkz7PkEIRTru3hVm5X7rktunYKviIiILLe8jes65HBp3nctcqc01UFEREREloKCr4iIiIgsBQVfERER+UZNliNLGGplMkHBV0RE5ItK+iF+zRpvlVukUmviVkbb6VYaPlH8pSoZMugFeF7E6TYNwwG9wMOLHv7GDcO4R8dr0NwDeEur4dHpxWdRuN/B84LRi8jDC0L6D/+25QopY4y57yJERESWStQg9XgLym2OAoccQBJSK/7Er/Eauz0f+5LNxT5bMiDqtHDXf+XduIb0IKLTcln/9R3l9hGBc5cFiNwPjfiKiIh8DTJFnMoj+PiaZmdwx23lsRwbe+YtC8e2F35E5Fug4CsiIvKV6cf6m7vIXVDwFRER+Sr0CP33wCPc4tTWCcM+fq2K23CxV1YoNUKmY3EcNqlWmwR+A6do4/r9s3msUZOK7eIFHtWSRaXVQ894yTLTBhYiIiL3Zkgy6NHvRXjNbX6Ln7Kx26JmTbbKHeA5FZLtiGYBhlaN/E8/4eQO6VTyxIFDwbOJAoc8kI8L/GW9SG4lppbvUHv8C0n7kIqdp5Lrk3rsUiyFVPI3rzhqpHi8dY0T611Mzbp5QyJ3QMFXRETkviR9ep0O7t9f8/HpKw4HFfJTh4dhi9rAoVMYvU4Xitj8yksvZFBZwa++xvZap58pVAO6hSErBYAVnBc7xCvTD6m9JY5hppFPZLnHHFevcWI6c/NGRO6Igq+IiMh9yRQoVl1aYcjfXteo+UV8J396uB95fKSI32wSjt9b2dlhhxz0IvyP4GSmAmY6j1WcvMhTrLokgw4tt8OQwe3UnM6QSV992lVSqdTnX+SB0oJa90fBV0RE5F7lsL2Azf5jfl13sFYi3PEI75CPgIXjuhTOfyzq8R6wkgSYN7o6JGqUcHpVQr9JPmrwyz/2Pr/cYUJynYnCVwRkhT+5D3q4TURE5L6lLbZbdVZ5xy/VBtE4WKbTj+B9h95g9vQkbOH/IU8ZeOnPPuzGIKAZxBA1sbfe4rjO58xsuCBqfsd3313jpxndYqsit0PBV0RE5CuQtlxa9VV4t0V1O2IIFIoVVvmdn50qXjQgSQZEXhXbz1P69zbuRhZ+q+I0OvTjhLjnU3X7FEs54rjHR6DfHwAJUTjapSweft72vVbNYMw1fvRgm3yFFHxFRES+oJmtcvemt8pNU6j5vHoK73+1sRsBvVyVzsEL1pKAnx//ie9WSjSTCkGrRIY0xWbE7kaBQfOv/Pn7AnYzoerVKKQhV6rxovwDez//iXxpm0GxxsYP8I9KBa8f0w8CAoAoIIhiSPoEwaiuKAi+4NbJIl+OtiwWERGRexLj29+zvvcDq0/ypIG4/5Z//Zjlxycr5IDh4C3v/q3MP/1LlT/8c42fX76H7FM2agWGYUQ/ybDiVKlVSuRv4aG7M0PiXkinNd1m6XTaSNzv4L/8nX/34v/kf0j+A9Wt3/m4Wqcb1rCm6kj6IR2vxvqv71jdbNOolCiuaMWLe2NERERE7sWRaZcfmc3uydRrDJRN+2j81knXbD4av+7WDWAot83ksDl8ZcpgWK2bg5MLDXy+eW2OHe8+M+X2oTHmcFw3JrvWNofnr3GybzbYMPt3UZ98Ek11EBERkfuTdrCtS4Zq0xZOJbf4eN6m+hx4t0UrvMZWz0mfMOwR38IWdhnLJpckQBrIks3Cx9fruP5g9sR0mhw50rc6Ii03oeArIiIi9ySH49e46jG4gtvCWZh9M2Tyo9/i66yz1g/46adtrpORr5SzaVUnC81ZNIM2a1nYW3doRNoc+muk4CsiIiIPWEIyGP2Wz33BubNRA9s/9wRg3qHlPSfLO7Zsl85thGu5VQq+IiIi8mANoxbbL4HVHSrFLzWXICEM/LlHMqUmQX0VPr6kUvVva788uSXauU1EREQelkFAsxkz7PkEIRTru3hVm5Uv0GYOIO7T8d+TvrCdHkAaq+bT7lmsv17HKeQJa3dZmHwKBV8RERF5WPI2ruuQw6V5xalRI8XjrYvvr3+fYn3qdbl9RLB4IvFUmyPVUhpn4VrHeZyWRxj+lZdbLs3i9hVVypei4CsiIiLfLMs95rg69UbU4Lu/9nn1wcOezrnpT5sfnC428IaXTK3IlGgGdXqPt9iyK2RxKX5SC3IXFHxFRETk25XOkJnOp5nM+O3M5NebXnj2uvPOsGr47R7W+ms+fk5Tcmv0cJuIiIh8JSZLgCUMH9RqYEOGyfya806T1lr2y5ckcyn4ioiIyL0bxj06XoPmHsBbWg2PTi8+i8L9Dp4XjF5EHl4Q0r/T5cKGxL0OnjdevSEKCDoX2xzGPYJGldrbt9SqDYKpmkdy2F5AffUua5XrShljzH0XISIiIvJFDDo0gxirUsH6gsv+ytdBwVdEREREloKmOoiIiMgX0serFKl4/eud7VUoVjyud/Y3ZDigF3h40dm8imHSJ/RbdAb3V9a3QMFXREREvoxhQj98S9hPuPrZtSFJP+Rt2Cd5UA+6fZ5kEOFvO/zlbz8TDEY3Hvc6eNUiP63/nd7CtYPlOjTVQURERORrEjVIPd6a2VQj9m2+X9+j3jXUrHuu7wHTiK+IiIiILAUFXxERkaUzJO4FeK7Dit0iaFWw8ilSuRVKbocBQNIn9Bs4Kw6toIWdT5GymvQAhn38WhW34WKvrFBqhEyv8hWHTarVJoHfwCnauH6fITAcRPiNClbF5/Qv9oMObsXF63TwXJui2xkdGw6I/AYVq4I/9ef9YT+g4dbwOgGtqoNTbTGZCpv0x/eUq+EFLqV8ilQqj93qXWNqxSIJ/dCn4azgtAJadp5UyqLZu3k/QELUrGC7HoFXpWRVaPWWaD7HfTIiIiKyZE7M8dGueQ6G7Jp5dXBsjDk2H16tmSyY1Z0Dc3J8ZHafYwCzWt83H/ZfmZ2dN+aDOTSvyqtm52B8pf1NkwXz9NWhMcaYo901ky23zeG4pYOdRwaypn4w1Wa5bY7GdexvYMrto8nZZudR3XSNmWq/bCaHTw7qZpWn5tXk4ubE7G9kDdk1s3tkjDGHpl3GwKp5vnt4dpzJ8Zt01bE52n1uAMNq3ex/2DevdnbMmw837Qdjjt88H93X+GB3E8OTV6fnmm7dwHS/GHPULhvA1Ls3vA8xxhijEV8REZGlkyaTy5EHsGxKhQyQYaVSo/YI3jU69DI5cnmAMm6lyEqxguuWyIctagOHYmF8pUIRG/jdCxkQ4VVfY1dLo2sDhWpAdz+kWphq89SQ4RD2mt541LZAxbdZAdKn7U8khK0t3pUrFE/fT1Os1Hj08TXbfg+Y7CFsU7HzQJoVywJe0x/ctKsy5MaFlN0KxZUiFdelGN+0HyCz4vBix2ElN9XO2xg9t3b3/njfBYiIiMjXooBVAn6NGMScC6kj/cjjI0X8ZpNw/N7Kzg475KAX4X8EJzO1M0Q6j1Vc1F6GUq3NmrXO4+9alHd8PNdi/r4SfXovgfJZvB01XqAI/CPsE7u5uZ+81LBHs1SiQY1Ox6WQvvoj8Jn9kC9SdRMGnRZuZ8hobol8CQq+IiIiciYDkCe3YFezIR8BC8d1KZw/GPV4D1hJMrnQ1fIOfj+PVa3Q+OUxK8HOggCahiwwGI2MnkbcdHr0e/qaifWWfFY/DCMaJYdeNcRv5okav/CPvTsvWdDDbSIiInIqIRkAz4oLRz7T6UfwvkNvcO6TYQv/D3nKwEt/9iEvBgHNYN4f8mM6zYA4Y+H6ffr7m+Te/kIrTOacu4JVycL7kP70pZKEAfC0VOAG472QLuCGMXF4/dFe+Lx+iJo2W28dXCd/k4rlMyj4ioiILLMkOQtngw6tl6vsuPbC8dpCscIqv/OzU8WLBiTJgMirYvt5Sv/ext3Iwm9VnEaHfpwQ93yqbp9iaX4sTcKz3cgyRZsSWTJzR2/TFKtN1rJ7NP3e2eejgGC1zvYXDpE37weIex+B/mjecRIRBgAxw2T4GatPyLXc99N1IiIich+6pg6GR0/M0/Jzs7mzYZ6VN0z7w7ExxpjjD22zuYqBrFl7sW/Gb4+OHbwwaz9mRysdZH80aztdc3b40OxuPDU/ZjHwg1l99sIcHBtjzLH5sLtpVscrSbzoHhljjsybjSfm6dqmab95Y15tPDVrLw7MiTHm+MPuVPtdM1nf4OTwjdl59sQ8q7fNm3bdPN9sn9Z2/OGVWcti4Kmp7x+Zk+MP5tXaqM6n9X1zo4Udjj+Y9uaqAUx27YXZn+qIm/WDMScHL0z5h9H7Tzfapru7YX4A80P5hTk4+mB2p9rrHhlzctQ1L8b3sbq5aw6Pzxcp16Wd20REROYaEvdCOq0aP798D9mnbNTOntKP+x38l79TmNpd69YlEa3aNs3gd/6VH3lq12g0KhSuOX32chGN1GO2ym2OAudm0wREHhg93CYiIjJXmlyhRKXSGwVfq0LNnQ6ILrVShcrwjv44PezRsLeh1iTchp7nUtn6mb8M4LBTmbvigohcTnN8RUREbihj2eSSeQ9ifb4kbBFVPWqlFXK5FUq1AO858HuToHflx6/Xxvj/Na9UloWCr4iIyE3lbFrVC4tZ3YqEItv29ASENIViGYg/P6gO+wStiNzODjvFmI7nEWn3BFkCmuogIiJyE1EDe1C5s/m9+ZJz4b1R4HWwVj7z4ukV7OrnXkTk4dGIr4iIyCdLCAP/kuMxvp0ilbr6x/avO9Sa0OvssbpTofhl92oQ+WZoxFdEROQ6BgHNZjx6uC3u0/Hfk144yyGH7R1zfJ3rpq+5RMMgoNWv02zdzdQKkWWg4CsiInIdeRt3alWHaimNc8lgbTqT4fYGZgf4tQ4V38e67KLDHs1SiQa1mW1/U6nUrVUiXyetTns9Cr4iIiI3kC428IaLU+gwSa73EFo6Q+bShDwkargMXJ9a/tNqnFAoEhlR8BUREbmR9CWBNSaofM/63tVXKV+6AcaQqFGimffwp4Z6h4M+SW6F3Pn20wXcMMa9RvUiy0jBV0RE5NblcALDxXUZPs3Ar9BIN2mVMiTj9YKHgw7bzSE1T6syiHwqBV8REZG5xlsWe+PVG6KAoJOjaBVZuZUtgy8XBw7W+ms+8pq9X2aPPds91s5tIjeQMpr4IyIiInemj1epEhZbeJWrR6n7XoVqWKTlVdCY9sgw6RN1QoZWlVIeYEjSj+iEQ6xqSf8I+gRax1dERETuzjChH74l7F/nYb8hST/kbdgn0T7KAMS9Dl61yE/rf6cXA8T0Oh7V4k+s/72HNtz7NBrxFRERkaU2jHtE/TSF4gpfYBbLJ4t9m+/X96h3DTULRhukfM/6Xp2uqWHdd4EPiEZ8RUREZKkl4TY//RTQv+9C5M4p+IqIiCyFIXEvwHMdVuwWQauClU+Ryq1QcjsMAJI+od/AWXFoBS3sfIqU1aQHMOzj16q4DRd7ZYVSIySZunocNqlWmwR+A6do4/p9hsBwEOE3KlgV/+zP8oMObsXF63TwXJui2xkdGw6I/AYVq8L0Ts7DfkDDreF1AlpVB6faIho3nvTH95Sr4QUupXyKVCqP3epdbx3lT+3FuEfQqlLK1/A7LsVcily1QwIM+z61qkvDtVlZKdEIp3ooGfVfzfPx3BLFSvP0Hhj28V2bSjMgaNhYJZfO4A6KFzAiIiKyBE7M8dGueQ6G7Jp5dXBsjDk2H16tmSyY1Z0Dc3J8ZHafYwCzWt83H/ZfmZ2dN+aDOTSvyqtm52B8pf1NkwXz9NWhMcaYo901ky23zeG4pYOdRwaypn4w1Wa5bY7GdexvYMrto8nZZudR3XSNmWq/bCaHTw7qZpWn5tXk4ubE7G9kDdk1s3tkjDGHpl3GwKp5vnt4dpzJ8asdtcsGRjVc2YvHB+bFEwxkzVr7wBzsvjA7r7rm+PCVKa/umINJjZtZw6TukwNTX31kNrsno4sc75pnYFjbNUen/bU5bn98Pxv75mSmPkz9tMCj8T1fr2Y5oxFfERGRpZAmk8uNVgCwbEqFDJBhpVKj9gjeNTr0MjlyeYAybqXISrGC65bIhy1qA4diYXylQhEb+N0LGRDhVV9jT60uUKgGdPdDqoWpNk8NGQ5hr+mNRzwLVHybFSB92v5EQtja4l25QvH0/TTFSo1HH1+z7ffgdGNom4qdB9KsWBbwmv7gVjpuRjqTI5MBcHGdAgW7ilsp0GvVGDhFCuMaC0Ub+B0vHBAH22wNKziTTUgyNs0P+3QbJXLASrHGzotzqzMM4pkRdbkdWsdXRERkqRWwSsCvEYOYuUtj9SOPjxTxm03C8XsrOzvskINehP8RnMzUY2HpPFZxUXsZSrU2a9Y6j79rUd7x8VxrwUNlfXovgfJZvB01XqAI/CPsE7uLdr1bZPJg2Pn393ic2pp55+xhsqv0ibyPUPRpNsNJkezs7EDu/6MX7EHeYbrSzErx9KG0dMHBLcREfoNGBMMEvsqn7L4BCr4iIiLLLgOQJ7cgbA35CFg4rjse0ZwS9XgPWMknpLW8g9/PY1UrNH55zEqwQ6fjUriwBXQassAgJoaz4JhOj35PL9wz+hI5bO+Y46l34qDCn39e4c3x7AoJ6WuHzyGjLnJw3fM9FOO3uHgP0wYBlWKDFb9D0xni21u8vG7T8kk01UFERGSpJSQD4FlxTvAcSacfwfsOvcG5T4Yt/D/kKQMv/dmH3RgENIN5q8zGdJoBccbC9fv09zfJvf2FVjjvD/srWJUsvA/pT18qSRgAT0uF+UHyCulMhsz0Txrg3HuZDNeP1WlGXdQbPSR4VihhK+S/sLLw3iPsTR8bEjVbRMT47t/4reDiWBrmvWsKviIiIssmSc5C6qBD6+UqO669cLy2UKywyu/87FTxogFJMiDyqth+ntK/t3E3svBbFafRoR8nxD2fqtunWJofS5OwdbpqQaZoUyJLZu7obZpitclado+mf5YakyggWK2z7eRvdv+3rkCxsgq//4xT9YgGCckgwqva+PkSf3MaPOU9v1TP+i9sOLTyNhYxgwgY9BgkwCAkiIBkSDLULh637r6frhMREZEvpWvqYHj0xDwtPzebOxvmWXnDtD8cG2OMOf7QNpur4xULXuyb8dujYwcvzNqPWQMYsj+atZ2uOTt8aHY3npofsxj4waw+e2EOjo0x5th82N00q+OVJF50j4wxR+bNxhPzdG3TtN+8Ma82npq1FwfmxBhz/GF3qv2umSzKcHL4xuw8e2Ke1dvmTbtunm+2T2s7/vDKrGUx8NTU94/MyfEH82ptVOfT+r65zsIO11/V4cQc7r8Yt7dqNtsH5miy9II5Ngcv1sZ9gMn+uGZ2umc9dHzwwjxb/WF87KnZ2D086703G+ZJ9qxf9188MVmy5snGG3N41DUvxvezurlrDo9PzFF3qobdw6nvQa6indtERESWRkQj9ZitcpujwLnRNIFvURJ5eFEO2y3NfbhPvh16uE1ERESWWsaq4Grf36WgOb4iIiJL5HSzsHutQuR+aKqDiIjIMhj2CbwOg3HiTWcyFEoVLM13kCWi4CsiIiIiS0FTHURERERkKSj4ioiIiMhSUPAVERERkaWg4CsiIiIiS0HBV0RERESWgoKviIiIiCwFBV8RERERWQoKviL/f7t1IAAAAAAgyN96kIsiAGBBfAEAWBBfAAAWxBcAgAXxBQBgQXwBAFgQXwAAFgLMjnZEHTTLegAAAABJRU5ErkJggg==\n","text/plain":["<IPython.core.display.Image object>"]},"execution_count":42,"metadata":{},"output_type":"execute_result"}],"source":["#提示：准确率 (Accuracy),精确度（Precision）,Recall,f-分数计算方法\n","Image('Snipaste_2020-01-05_16-39-27.png')"]},{"cell_type":"markdown","metadata":{"id":"dgXU3WIdcQfo"},"source":["#### 提示5\n","* 混淆矩阵的方法在sklearn中的`sklearn.metrics`模块\n","https://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html?highlight=confusion_matrix#sklearn.metrics.confusion_matrix\n","\n","* 混淆矩阵需要输入真实标签和预测标签\n","* 精确率、召回率以及f-分数可使用`classification_report`模块\n","\n","https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html?highlight=classification_report#sklearn.metrics.classification_report"]},{"cell_type":"code","execution_count":16,"metadata":{"id":"V2JuViDQcQfp","executionInfo":{"status":"ok","timestamp":1680522603463,"user_tz":-540,"elapsed":527,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["#写入代码\n","from sklearn.metrics import confusion_matrix"]},{"cell_type":"code","execution_count":17,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":75},"id":"Kia6jQ1BcQfp","executionInfo":{"status":"ok","timestamp":1680522645778,"user_tz":-540,"elapsed":267,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"070a392e-0eb8-4252-e5a8-7f347a9a14eb"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["RandomForestClassifier(max_depth=5)"],"text/html":["<style>#sk-container-id-3 {color: black;background-color: white;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>RandomForestClassifier(max_depth=5)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">RandomForestClassifier</label><div class=\"sk-toggleable__content\"><pre>RandomForestClassifier(max_depth=5)</pre></div></div></div></div></div>"]},"metadata":{},"execution_count":17}],"source":["#写入代码\n","rfc = RandomForestClassifier(n_estimators=100, max_depth=5)\n","rfc.fit(X_train, y_train)"]},{"cell_type":"code","execution_count":18,"metadata":{"id":"VE7Bv7xfcQfp","executionInfo":{"status":"ok","timestamp":1680522666776,"user_tz":-540,"elapsed":427,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["#写入代码\n","pred = rfc.predict(X_train)"]},{"cell_type":"code","execution_count":19,"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"AwqpmcsOcQfq","executionInfo":{"status":"ok","timestamp":1680522691081,"user_tz":-540,"elapsed":495,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"e8d3c088-9c23-412c-fcae-cc024fc024d1"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["array([[384,  28],\n","       [ 68, 188]])"]},"metadata":{},"execution_count":19}],"source":["#写入代码\n","confusion_matrix(y_train, pred)"]},{"cell_type":"code","source":["from sklearn.metrics import classification_report"],"metadata":{"id":"V9H596Ai9bbn","executionInfo":{"status":"ok","timestamp":1680522747253,"user_tz":-540,"elapsed":392,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"execution_count":20,"outputs":[]},{"cell_type":"code","source":["#精确率，召回率以及f1-score\n","print(classification_report(y_train, pred))"],"metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"OybQv6yJ9iyx","executionInfo":{"status":"ok","timestamp":1680522823810,"user_tz":-540,"elapsed":394,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"7a809026-bacd-4ee2-ed4e-e0a23da280f4"},"execution_count":21,"outputs":[{"output_type":"stream","name":"stdout","text":["              precision    recall  f1-score   support\n","\n","           0       0.85      0.93      0.89       412\n","           1       0.87      0.73      0.80       256\n","\n","    accuracy                           0.86       668\n","   macro avg       0.86      0.83      0.84       668\n","weighted avg       0.86      0.86      0.85       668\n","\n"]}]},{"cell_type":"markdown","metadata":{"id":"H0_RHCSIcQfq"},"source":["【思考】\n","* 如果自己实现混淆矩阵的时候该注意什么问题"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"OIAwe-kBcQfq"},"outputs":[],"source":["#思考回答\n","# https://blog.csdn.net/zhangxiaohua18/article/details/122311808"]},{"cell_type":"markdown","metadata":{"id":"cniCfLSlcQfq"},"source":["#### 任务三：ROC曲线\n","* 绘制ROC曲线"]},{"cell_type":"markdown","metadata":{"id":"1rcMiaHLcQfr"},"source":["【思考】什么是ROC曲线，OCR曲线的存在是为了解决什么问题？"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"qLidwPcWcQfr"},"outputs":[],"source":["#思考\n","# https://blog.csdn.net/sinat_33209811/article/details/125755163"]},{"cell_type":"markdown","metadata":{"id":"c1GJdb6ncQfr"},"source":["#### 提示6\n","* ROC曲线在sklearn中的模块为`sklearn.metrics`\n","\n","https://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_curve.html?highlight=roc_curve#sklearn.metrics.roc_curve\n","* ROC曲线下面所包围的面积越大越好"]},{"cell_type":"code","execution_count":22,"metadata":{"id":"nfHoYONCcQfs","executionInfo":{"status":"ok","timestamp":1680523185531,"user_tz":-540,"elapsed":864,"user":{"displayName":"张婷","userId":"16421421487827652066"}}},"outputs":[],"source":["#写入代码\n","from sklearn.metrics import roc_curve"]},{"cell_type":"code","execution_count":23,"metadata":{"colab":{"base_uri":"https://localhost:8080/","height":1000},"id":"1aqYCseAcQfs","executionInfo":{"status":"ok","timestamp":1680523474574,"user_tz":-540,"elapsed":1205,"user":{"displayName":"张婷","userId":"16421421487827652066"}},"outputId":"97134ddf-5a6f-4c9a-eddb-bf6443045b0b"},"outputs":[{"output_type":"execute_result","data":{"text/plain":["<matplotlib.legend.Legend at 0x7f5811b88b20>"]},"metadata":{},"execution_count":23},{"output_type":"stream","name":"stderr","text":["WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n","WARNING:matplotlib.font_manager:findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei\n"]},{"output_type":"display_data","data":{"text/plain":["<Figure size 1000x600 with 1 Axes>"],"image/png":"iVBORw0KGgoAAAANSUhEUgAAA04AAAINCAYAAAAJGy/3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABE2ElEQVR4nO3deXgUZdr+/bMT0lmARBggEAiLIKsQZMsAowwaDSPDDGqUAX6IC7jhgkEUZAngCMrI4mggCAKOA4L74yMIOlFGhSgQCeoDgggIKsEFSDCEdJb7/cOX1oYklQ69Jt/PcfRxpKvuqr4KCuiTu+oqmzHGCAAAAABQoRB/FwAAAAAAgY7gBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAW6vi7AF8rKyvTd999p/r168tms/m7HAAAAAB+YozRyZMnFRcXp5CQyueUal1w+u677xQfH+/vMgAAAAAEiMOHD6tFixaVjql1wal+/fqSfvnFiY6O9nM1AAAAAPwlPz9f8fHxzoxQmVoXnM5cnhcdHU1wAgAAAFClW3hoDgEAAAAAFghOAAAAAGCB4AQAAAAAFghOAAAAAGCB4AQAAAAAFghOAAAAAGCB4AQAAAAAFghOAAAAAGCB4AQAAAAAFghOAAAAAGCB4AQAAAAAFghOAAAAAGCB4AQAAAAAFvwanN5//30NGTJEcXFxstlsev311y232bRpk3r06KHw8HC1a9dOK1eu9HqdAAAAAGo3vwangoICJSQkKD09vUrjDxw4oMGDB2vgwIHKycnR+PHjNWbMGG3cuNHLlQIAAACozer488P/9Kc/6U9/+lOVx2dkZKhNmzaaN2+eJKlTp0768MMPtWDBAiUnJ3urTAAAAAC1nF+Dk7uysrKUlJTksiw5OVnjx4/3T0EAAABADWaMUWFxqVf2HRkWKpvN5pV9e0NQBafc3FzFxsa6LIuNjVV+fr4KCwsVGRl5zjZFRUUqKipyvs/Pz/d6nQAAAECwM8YoJSNL2V8f98r+d81KVpQ9eOJIje+qN2fOHMXExDhf8fHx/i4JAAAACHiFxaVeC03BKHginqSmTZvq6NGjLsuOHj2q6OjocmebJGny5MlKTU11vs/Pzyc8AQAAAG7YPjVJUfZQj+4zMsyz+/O2oApOffv21fr1612WvfPOO+rbt2+F24SHhys8PNzbpQEAAAA1VpQ9NKguq/MGvx79zz//rH379jnfHzhwQDk5OWrYsKFatmypyZMn69tvv9W//vUvSdIdd9yhp59+Wg8++KBuueUWvfvuu3rxxRe1bt06fx0CAAAAgpg3mx8Eu1MOfl1+y6/Bafv27Ro4cKDz/ZlL6kaPHq2VK1fqyJEjOnTokHN9mzZttG7dOt1///168skn1aJFCy1btoxW5AAAAHCbt5sfoGaxGWOMv4vwpfz8fMXExCgvL0/R0dH+LgcAAAB+cspRos7TN/q7jIDXq1UDvXRH36BqHV5V7mSD2n2hIgAAACDvND+oKYLteUveQnACAABAQPL2/Ue/vYeH5gewwtkBAACAgMP9Rwg0Nf4BuAAAAAg+vnz4aq9WDYLumULwPWacAAAAENC8ff8R9/CgKghOAAAACGjcf4RAwBkIAACACvnrAbE8fBWBhuAEAACActGgAfgVzSEAAABQLl82aKgIjRsQKJhxAgAAgCV/PSCWxg0IFAQnAAAAWKJBA2o7zn4AAIAa7HyaO9CgAfgVwQkAAKCGorkD4Dk0hwAAAKihPNXcgQYNADNOAAAAtcL5NHegQQNAcAIAAKgVaO4AnB/+9AAAANQQZzeCoLkD4DkEJwAAgBqARhCAd9EcAgAAoAaorBEEzR2A88eMEwAAQA1zdiMImjsA54/gBAAAUMPQCALwPP5EAQCAoHF28wP8ikYQgHcRnAAAQFCg+QEAf6I5BAAACAqVNT/Ar2gEAXgHM04AACDonN38AL+iEQTgHQQnAAAQdGh+AMDXuFQPAAAAACwQnAAAAADAAsEJAAAAACwQnAAAAADAAsEJAAAAACzQjgYAgCBljFFhcam/y/CZU47ac6wAAg/BCQCAIGSMUUpGFg+EBQAf4VI9AACCUGFxaa0NTb1aNVBkGA+/BeBbzDgBABDktk9NUpS99gSJyLBQ2Ww2f5cBoJYhOAEAEOSi7KGKsvNPOgB4E3/LAgAQQKra8IFGCQDgWwQnAAACBA0fACBw0RwCAIAAUZ2GDzRKAADfYMYJAIAAVNWGDzRKAADfIDgBABCAaPgAAIGFv5EBAPCBqjR9oOEDAAQughMAAF5G0wcACH40hwAAwMvcbfpAwwcACDzMOAEA4ENVafpAwwcACDwEJwAAfIimDwAQnLhUDwAAAAAsEJwAAAAAwALBCQAAAAAsEJwAAAAAwAJ3pwIAcJaqPKzWHTzYFgCCH8EJAIDf4GG1AIDycKkeAAC/4e7Dat3Bg20BIHgx4wQAQAWq8rBad/BgWwAIXgQnAAAqwMNqAQBn8K8BAMCnPN14wdNo5AAAKA/BCQDgMzReAAAEK5pDAAB8xpuNFzyNRg4AgN9ixgkA4BeebrzgaTRyAAD8FsEJAOAXNF4AAAQT/sUCAHhEVZo+0HgBABCsCE4AgPNG0wcAQE1HcwgAwHlzt+kDjRcAAMGGGScAgEdVpekDjRcAAMGG4AQA8CiaPgAAaiL+ZQMAOFWlwUN5aPoAAKjpCE4AAEk0eAAAoDI0hwAASHK/wUN5aPoAAKipmHECAJyjKg0eykPTBwBATUVwAgCcgwYPAAC44l9FAE7VbQyAmoEGDwAAVIzgBEASjQEAAAAqQ3MIAJI80xgANQMNHgAAOJffZ5zS09P1j3/8Q7m5uUpISNBTTz2lPn36VDh+4cKFWrx4sQ4dOqRGjRopJSVFc+bMUUREhA+rBmq26jYGQM1AgwcAAM7l1+C0du1apaamKiMjQ4mJiVq4cKGSk5O1Z88eNWnS5Jzxq1ev1qRJk7R8+XL169dPe/fu1U033SSbzab58+f74QiAmonGAAAAAK78eqne/PnzNXbsWN18883q3LmzMjIyFBUVpeXLl5c7fsuWLerfv79GjBih1q1b66qrrtLw4cO1detWH1cOAAAAoDbxW3ByOBzKzs5WUlLSr8WEhCgpKUlZWVnlbtOvXz9lZ2c7g9L+/fu1fv16XX311T6pGQAAAEDt5LdrcX788UeVlpYqNjbWZXlsbKy++OKLcrcZMWKEfvzxR/3hD3+QMUYlJSW644479PDDD1f4OUVFRSoqKnK+z8/P98wBAAAAAKg1gqqr3qZNmzR79mwtWrRIn3zyiV599VWtW7dOjzzySIXbzJkzRzExMc5XfHy8DysGAAAAUBP4bcapUaNGCg0N1dGjR12WHz16VE2bNi13m2nTpmnUqFEaM2aMJKlr164qKCjQbbfdpilTpigk5NwcOHnyZKWmpjrf5+fnE54AAAAAuMVvM052u109e/ZUZmamc1lZWZkyMzPVt2/fcrc5derUOeEoNPSXlsnGmHK3CQ8PV3R0tMsLqG2MMTrlKLF4lfq7TAAAgIDl137DqampGj16tHr16qU+ffpo4cKFKigo0M033yxJuvHGG9W8eXPNmTNHkjRkyBDNnz9fl1xyiRITE7Vv3z5NmzZNQ4YMcQYoAK6MMUrJyOLhtgAAAOfBr8Fp2LBh+uGHHzR9+nTl5uaqe/fu2rBhg7NhxKFDh1xmmKZOnSqbzaapU6fq22+/VePGjTVkyBA9+uij/joEIOAVFpe6FZp6tWqgyDD+IwIAAOC3bKaia9xqqPz8fMXExCgvL4/L9lArnHKUqPP0jZKk7VOTFGWvPBRFhoXKZrP5ojQAAAC/cicb+HXGCYBvRdlDFWXnjz0AAIC7+AYF1DDGGBUW/9rogaYPAAAA54/gBNQgNIIAAADwjqB6AC6AylXWCIKmDwAAANXHjBNQQ53dCIKmDwAAANVHcAKC1Nn3Mkmu9zPRCAIAAMBz+FYFBCHuZQIAAPAt7nECgpDVQ225nwkAAMCzmHECglx5D7XlfiYAAADPIjgBQY57mQAAALyPS/UAAAAAwALBCQAAAAAsEJwAAAAAwALBCQAAAAAsEJwAAAAAwAKtuIAgYIxRYXGp8/0pR2klowEAAOBpBCcgwBljlJKRVekDbwEAAOBdXKoHBLjC4tIKQ1OvVg0UGRZa7joAAAB4DjNOQBDZPjVJUfZfg1JkWKhsNpsfKwIAAKgdCE5AEImyhyrKzh9bAAAAX+NSPQAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAt1/F0AUJsZY1RYXFrpmFOOytcDAADA+whOgJ8YY5SSkaXsr4/7uxQAAABY4FI9wE8Ki0vdCk29WjVQZFioFysCAABARZhxAgLA9qlJirJXHooiw0Jls9l8VBEAAAB+i+AEBIAoe6ii7PxxBAAACFR8UwO8gKYPAAAANQvBCfAwmj4AAADUPDSHADyMpg8AAAA1DzNOgBfR9AEAAKBmIDgBXkTTBwAAgJqBb3TAeTq7EQRNHwAAAGoeghNwHmgEAQAAUDvQHAI4D5U1gqDpAwAAQM3BjBPgIWc3gqDpAwAAQM1BcALcUNn9TDSCAAAAqLn4lgdUEfczAQAA1F7c4wRUEfczAQAA1F7MOAHVwP1MAAAAtQvBCUFt586dWrJkiXJycnTy5EnVr19f3bt31+23366EhASvfS73MwEAANQufPNDUNq2bZvGjx+vLVu2nLMuKytLixcvVv/+/bVgwQL17t270n2d3fChIjzYFgAAoPYiOCHorFu3TikpKTp9+rTL8oiICJdlmzdv1mWXXaaXX35ZgwcPLndfNHwAAABAVdAcAkFl27ZtLqGpQ4cOWrRokfLy8lRYWKgTJ05o0aJF6tChgyTp9OnTSklJ0bZt28rdX2UNHypCIwgAAIDax2aMMf4uwpfy8/MVExOjvLw8RUdH+7scuKl///7Oy/NuuOEGPf/887Lb7eeMKyoq0qhRo/TSSy85t/vwww/PGXfKUaLO0zdKOrfhQ0VoBAEAAFAzuJMNmHFC0MjJyXGGpg4dOlQYmiQpPDxc//73v50zT5s3b9bOnTsr3f+Zhg9WL0ITAABA7UNwQtB45plnnD/fd999LqHJGKNTjhKXV4lCdMdd45xjFi3OOGcMDR8AAABQFTSHQNDIyclx/jxixAjnz5U1eCgrinX+/Nyb/9XGCzZ6tUYAAADUTMw4IWicPHlS0i/d82JiYpzLK2vwEBJeVwoNkySVOQor3DcNHwAAAFAZZpwQNOrXry/pl055eXl5LuHpjLMbPOTl5Snu8WJJUs92zZQ5K7ncfdPwAQAAAJVhxglBo3v37s6fV69eXe6Ysxs8vPbSWue6HpdcQsMHAAAAVAvBCUHjtttuc/785JNPyuFwVDq+qKhI//znP8vdHgAAAHAHwQlBo3v37urXr58kac+ePRo1alSF4enMc5z27Nkj6ZfnOCUkJPisVgAAANQsBCcElYULFyoiIkKS9OKLL6pbt256JmOxyooKJP1yT9OiRYuUkJDgfPhtZGSkFixY4LeaAQAAEPxsxhjj7yJ8yZ2nAyMwrVu3TikpKTp9+rTritAwqbTYZVFkZKReeuklDR482IcVAgAAIBi4kw2YcULQGTx4sN5//33179/fdcVZoal///7673//S2gCAADAeaMdOYJS79699eGHH2rnzp1atDhDz735X5U5CtWzXTP1uOQS3XbbbdzTBAAAAI/hUj0EvVOOEnWevlGStGtWsqLs/H8AAAAArHGpHgAAAAB4EMEJAAAAACwQnAAAAADAAsEJAAAAACwQnAAAAADAAsEJAAAAACwQnAAAAADAgt+DU3p6ulq3bq2IiAglJiZq69atlY4/ceKExo0bp2bNmik8PFzt27fX+vXrfVQtAAAAgNrIr08KXbt2rVJTU5WRkaHExEQtXLhQycnJ2rNnj5o0aXLOeIfDoSuvvFJNmjTRyy+/rObNm+vrr7/WBRdc4Pvi4RPGGBUWl1Y65pSj8vUAAADA+fJrcJo/f77Gjh2rm2++WZKUkZGhdevWafny5Zo0adI545cvX65jx45py5YtCgsLkyS1bt3alyXDh4wxSsnIUvbXx/1dCgAAAGo5v12q53A4lJ2draSkpF+LCQlRUlKSsrKyyt3mjTfeUN++fTVu3DjFxsbq4osv1uzZs1VayoxDTVRYXOpWaOrVqoEiw0K9WBEAAABqK7/NOP34448qLS1VbGysy/LY2Fh98cUX5W6zf/9+vfvuuxo5cqTWr1+vffv26a677lJxcbHS0tLK3aaoqEhFRUXO9/n5+Z47CPjM9qlJirJXHooiw0Jls9l8VBEAAABqE79equeusrIyNWnSRM8884xCQ0PVs2dPffvtt/rHP/5RYXCaM2eOZs6c6eNK4WlR9lBF2YPqdAUAAEAN4rdL9Ro1aqTQ0FAdPXrUZfnRo0fVtGnTcrdp1qyZ2rdvr9DQX2ceOnXqpNzcXDkcjnK3mTx5svLy8pyvw4cPe+4g4DHGGJ1ylJz14hJMAAAABAa//Re+3W5Xz549lZmZqaFDh0r6ZUYpMzNTd999d7nb9O/fX6tXr1ZZWZlCQn7JfHv37lWzZs1kt9vL3SY8PFzh4eFeOQZ4Bk0gAAAAEOj8+hyn1NRULV26VM8995x2796tO++8UwUFBc4uezfeeKMmT57sHH/nnXfq2LFjuu+++7R3716tW7dOs2fP1rhx4/x1CPAAqyYQNH0AAACAv/n1ppFhw4bphx9+0PTp05Wbm6vu3btrw4YNzoYRhw4dcs4sSVJ8fLw2btyo+++/X926dVPz5s1133336aGHHvLXIcDDymsCQdMHAAAA+JvNGGP8XYQv5efnKyYmRnl5eYqOjvZ3OZB0ylGiztM3SpJ2zUqmCQQAAAB8wp1sUO1vqMXFxcrNzdWpU6fUuHFjNWzYsLq7AgAAAICA5tY9TidPntTixYs1YMAARUdHq3Xr1urUqZMaN26sVq1aaezYsdq2bZu3agUAAAAAv6hycJo/f75at26tFStWKCkpSa+//rpycnK0d+9eZWVlKS0tTSUlJbrqqqs0aNAgffnll96sGwAAAAB8psqX6m3btk3vv/++unTpUu76Pn366JZbblFGRoZWrFihDz74QBdddJHHCgUAAAAAf6lycHrhhReqNC48PFx33HFHtQsCAAAAgEDj1+c4AQAAAEAwqPKM07XXXlvlnb766qvVKgYAAAAAAlGVg1NMTIw36wAAAACAgFXl4LRixQpv1gEAAAAAAavaD8AFqssYo8LiUuf7U47SSkYDAAAA/lfl4HTJJZfIZrNVaewnn3xS7YJQsxljlJKRpeyvj/u7FAAAAKDKqhychg4d6sUyUFsUFpdWGJp6tWqgyLBQH1cEAAAAWKtycEpLS/NmHaiFtk9NUpT916AUGRZa5VlNAAAAwJe4xwl+E2UPVZSdUxAAAACBr1rfWktLS7VgwQK9+OKLOnTokBwOh8v6Y8eOeaQ4BJezmz6Uh0YQAAAACEbVCk4zZ87UsmXLNGHCBE2dOlVTpkzRwYMH9frrr2v69OmerhFBgKYPAAAAqMlCqrPRqlWrtHTpUk2YMEF16tTR8OHDtWzZMk2fPl0fffSRp2tEEKis6UN5aAQBAACAYFKtGafc3Fx17dpVklSvXj3l5eVJkv785z9r2rRpnqsOQenspg/loREEAAAAgkm1ZpxatGihI0eOSJLatm2rt99+W5K0bds2hYeHe646BKUzTR8qexGaAAAAEEyqFZyuueYaZWZmSpLuueceTZs2TRdddJFuvPFG3XLLLR4tEAAAAAD8rVqX6j322GPOn4cNG6ZWrVppy5YtuuiiizRkyBCPFQcAAAAAgcAjD9H5/e9/r9///vee2BUAAAAABJxqXao3Z84cLV++/Jzly5cv1+OPP37eRQEAAABAIKlWcFqyZIk6dux4zvIuXbooIyPjvIsCAAAAgEBS7XbkzZo1O2d548aNnd32ULMZY1RYXOp8f8pRWsloAAAAILhVKzjFx8dr8+bNatOmjcvyzZs3Ky4uziOFIXAZY5SSkeXWA28BAACAYFat4DR27FiNHz9excXFuvzyyyVJmZmZevDBBzVhwgSPFojAU1hcWmFo6tWqgSLDKn/4LQAAABBsqhWcJk6cqJ9++kl33XWXHA6HJCkiIkIPPfSQJk+e7NECEdi2T01SlP3XoBQZFsrDbQEAAFDjVCs42Ww2Pf7445o2bZp2796tyMhIXXTRRQoPD/d0fQhwUfZQRdk90tUeAAAACFjV6qp3Rm5uro4dO6a2bdsqPDxcxhhP1QUAAAAAAaNawemnn37SFVdcofbt2+vqq692dtK79dZbuccJAAAAQI1TreB0//33KywsTIcOHVJUVJRz+bBhw7RhwwaPFQcAAAAAgaBaN6e8/fbb2rhxo1q0aOGy/KKLLtLXX3/tkcIAAAAAIFBUa8apoKDAZabpjGPHjtEgAgAAAECNU63gdOmll+pf//qX873NZlNZWZnmzp2rgQMHeqw4+J4xRqccJRavUn+XCQAAAPhUtS7Vmzt3rq644gpt375dDodDDz74oP7v//5Px44d0+bNmz1dI3zEGKOUjKwKH24LAAAA1FbVmnG6+OKLtXfvXv3hD3/QX//6VxUUFOjaa6/Vjh071LZtW0/XCB8pLC51KzT1atVAkWGh1gMBAACAIOf2jFNxcbEGDRqkjIwMTZkyxRs1IQBsn5qkKHvloSgyLFQ2m81HFQEAAAD+43ZwCgsL06effuqNWhBAouyhirJX60pOAAAAoMap1qV6/+///T89++yznq4FPlR+EwiaPgAAAADlqdaUQklJiZYvX67//Oc/6tmzp+rWreuyfv78+R4pDt5BEwgAAADAPdUKTp9//rl69OghSdq7d6/LOu55CXxWTSBo+gAAAAC4qlZweu+99zxdB/ykvCYQNH0AAAAAXHH3fw1jjFFhceX3Kv32XiaaQAAAAADWqvyN+Y477tDUqVPVokULy7Fr165VSUmJRo4ceV7FwT3cuwQAAAB4R5WDU+PGjdWlSxf1799fQ4YMUa9evRQXF6eIiAgdP35cu3bt0ocffqg1a9YoLi5OzzzzjDfrRjl4gC0AAADgHTZjjKnq4KNHj2rZsmVas2aNdu3a5bKufv36SkpK0pgxYzRo0CCPF+op+fn5iomJUV5enqKjo/1djkedcpSo8/SNkniALQAAAGDFnWzg1s0tsbGxmjJliqZMmaLjx4/r0KFDKiwsVKNGjdS2bVu+hAcQ7l0CAAAAPKfa36wbNGigBg0aeLIWAAAAAAhIIZ7c2auvvqpu3bp5cpcAAAAA4HduB6clS5YoJSVFI0aM0McffyxJevfdd3XJJZdo1KhR6t+/v8eLBAAAAAB/cis4PfbYY7rnnnt08OBBvfHGG7r88ss1e/ZsjRw5UsOGDdM333yjxYsXe6tWAAAAAPALt+5xWrFihZYuXarRo0frgw8+0IABA7Rlyxbt27dPdevW9VaNAAAAAOBXbs04HTp0SJdffrkk6dJLL1VYWJhmzpxJaAIAAABQo7kVnIqKihQREeF8b7fb1bBhQ48XBQAAAACBxO125NOmTVNUVJQkyeFw6O9//7tiYmJcxsyfP98z1QEAAABAAHArOF122WXas2eP832/fv20f/9+lzE8BBcAAABATeNWcNq0aZOXygAAAACAwOX2pXr5+fn6+OOP5XA41KdPHzVu3NgbdaEKjDEqLC51vj/lKK1kNAAAAIDqcis45eTk6Oqrr1Zubq4kqX79+nrxxReVnJzsleJQMWOMUjKylP31cX+XAgAAANR4bnXVe+ihh9SmTRtt3rxZ2dnZuuKKK3T33Xd7qzZUorC4tMLQ1KtVA0WGhfq4IgAAAKDmcmvGKTs7W2+//bZ69OghSVq+fLkaNmyo/Px8RUdHe6VAWNs+NUlR9l+DUmRYKE06AAAAAA9yKzgdO3ZMLVq0cL6/4IILVLduXf30008EJz+Ksocqyu727WoAAAAAqsjtb9u7du1y3uMk/XKvze7du3Xy5Ennsm7dunmmOgAAAAAIAG4HpyuuuELGGJdlf/7zn2Wz2WSMkc1mU2kp3d0AAAAA1BxuBacDBw54qw4AAAAACFhuBafnnntODzzwgKKiorxVDwAAAAAEHLfakc+cOVM///yzt2oBAAAAgIDk1ozT2fc2wXeMMSos/vXesVMO7iMDAAAAfMXt5hA8H8j3jDFKyciq8IG3AAAAALzL7eDUvn17y/B07NixaheEcxUWl1YYmnq1aqDIsNBy1wEAAADwDLeD08yZMxUTE+ONWlAF26cmKcr+a1CKDAtlFhAAAADwMreD09/+9jc1adLEG7WgCqLsoYqyu/3bBgAAAOA8uNVVz1szG+np6WrdurUiIiKUmJiorVu3Vmm7NWvWyGazaejQoV6pCwAAAAAkN4OTN7rqrV27VqmpqUpLS9Mnn3yihIQEJScn6/vvv690u4MHD+qBBx7QpZde6vGaAAAAAOC33ApOZWVlHr9Mb/78+Ro7dqxuvvlmde7cWRkZGYqKitLy5csr3Ka0tFQjR47UzJkzdeGFF3q0HgAAAAA4m1vBydMcDoeys7OVlJTkXBYSEqKkpCRlZWVVuN2sWbPUpEkT3Xrrrb4oEwAAAEAt59cuAz/++KNKS0sVGxvrsjw2NlZffPFFudt8+OGHevbZZ5WTk1OlzygqKlJRUZHzfX5+frXrBQAAAFA7+XXGyV0nT57UqFGjtHTpUjVq1KhK28yZM0cxMTHOV3x8vJerBAAAAFDT+HXGqVGjRgoNDdXRo0ddlh89elRNmzY9Z/xXX32lgwcPasiQIc5lZWVlkqQ6depoz549atu2rcs2kydPVmpqqvN9fn4+4QkAAACAW/wanOx2u3r27KnMzExnS/GysjJlZmbq7rvvPmd8x44d9dlnn7ksmzp1qk6ePKknn3yy3EAUHh6u8PBwr9QPAAAAoHbw+5NUU1NTNXr0aPXq1Ut9+vTRwoULVVBQoJtvvlmSdOONN6p58+aaM2eOIiIidPHFF7tsf8EFF0jSOcsBAAAAwFP8HpyGDRumH374QdOnT1dubq66d++uDRs2OBtGHDp0SCEhQXUrFgAAAIAaxma88VTbAJafn6+YmBjl5eUpOjra3+VUySlHiTpP3yhJ2jUrWVF2v+ddAAAAIOi5kw2YygEAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAACwQnAAAAALBAcAIAAAAAC3X8XQDOZYxRYXGp8/0pR2klowEAAAB4G8EpwBhjlJKRpeyvj/u7FAAAAAD/Py7VCzCFxaUVhqZerRooMizUxxUBAAAAYMYpgG2fmqQo+69BKTIsVDabzY8VAQAAALUTwSmARdlDFWXntwgAAADwNy7VAwAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALBCcAAAAAsEBwAgAAAAALARGc0tPT1bp1a0VERCgxMVFbt26tcOzSpUt16aWXqkGDBmrQoIGSkpIqHQ8AAAAA58vvwWnt2rVKTU1VWlqaPvnkEyUkJCg5OVnff/99ueM3bdqk4cOH67333lNWVpbi4+N11VVX6dtvv/Vx5QAAAABqC5sxxvizgMTERPXu3VtPP/20JKmsrEzx8fG65557NGnSJMvtS0tL1aBBAz399NO68cYbLcfn5+crJiZGeXl5io6OPu/6Pe2Uo0Sdp2+UJO2alawoex0/VwQAAADUTO5kA7/OODkcDmVnZyspKcm5LCQkRElJScrKyqrSPk6dOqXi4mI1bNjQW2UCAAAAqOX8Op3x448/qrS0VLGxsS7LY2Nj9cUXX1RpHw899JDi4uJcwtdvFRUVqaioyPk+Pz+/+gUDAAAAqJX8fo/T+Xjssce0Zs0avfbaa4qIiCh3zJw5cxQTE+N8xcfH+7hKAAAAAMHOr8GpUaNGCg0N1dGjR12WHz16VE2bNq102yeeeEKPPfaY3n77bXXr1q3CcZMnT1ZeXp7zdfjwYY/UDgAAAKD28Gtwstvt6tmzpzIzM53LysrKlJmZqb59+1a43dy5c/XII49ow4YN6tWrV6WfER4erujoaJcXAAAAALjD7y3bUlNTNXr0aPXq1Ut9+vTRwoULVVBQoJtvvlmSdOONN6p58+aaM2eOJOnxxx/X9OnTtXr1arVu3Vq5ubmSpHr16qlevXp+Ow4AAAAANZffg9OwYcP0ww8/aPr06crNzVX37t21YcMGZ8OIQ4cOKSTk14mxxYsXy+FwKCUlxWU/aWlpmjFjhi9LBwAAAFBL+P05Tr7Gc5wAAAAASEH0HCcAAAAACAYEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAt1/F1AbWeMUWFxqfP9KUdpJaMBAAAA+APByY+MMUrJyFL218f9XQoAAACASnCpnh8VFpdWGJp6tWqgyLBQH1cEAAAAoDzMOAWI7VOTFGX/NShFhoXKZrP5sSIAAAAAZxCcAkSUPVRRdn47AAAAgEDEpXoAAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYKGOvwsAAAAAzlZWViaHw+HvMlAD2O12hYSc/3wRwQkAAAABxeFw6MCBAyorK/N3KagBQkJC1KZNG9nt9vPaD8EJAAAAAcMYoyNHjig0NFTx8fEemSlA7VVWVqbvvvtOR44cUcuWLWWz2aq9L4ITAAAAAkZJSYlOnTqluLg4RUVF+bsc1ACNGzfWd999p5KSEoWFhVV7P0R4AAAABIzS0lJJOu/LqoAzzpxLZ86t6iI4AQAAIOCczyVVwG956lwiOAEAAACABYITAAAAAFggOAEAAADn6aabbpLNZpPNZlNYWJjatGmjBx98UKdPnz5n7JtvvqkBAwaofv36ioqKUu/evbVy5cpy9/vKK6/oj3/8o2JiYlSvXj1169ZNs2bN0rFjxyqt57333tPVV1+t3/3ud4qKilLnzp01YcIEffvtt5443FqJ4AQAAAB4wKBBg3TkyBHt379fCxYs0JIlS5SWluYy5qmnntJf//pX9e/fXx9//LE+/fRT/e1vf9Mdd9yhBx54wGXslClTNGzYMPXu3VtvvfWWPv/8c82bN087d+7U888/X2EdS5YsUVJSkpo2bapXXnlFu3btUkZGhvLy8jRv3rxqH19tfyAx7cgBAAAQsIwxKiw+v25o1RUZFupWY4Hw8HA1bdpUkhQfH6+kpCS98847evzxxyVJhw8f1oQJEzR+/HjNnj3bud2ECRNkt9t177336vrrr1diYqK2bt2q2bNna+HChbrvvvucY1u3bq0rr7xSJ06cKLeGb775Rvfee6/uvfdeLViwwGW7yy67zLndjBkz9PrrrysnJ8c5ZuHChVq4cKEOHjwo6ZdZtBMnTqh3795KT09XeHi4hg8frszMTH388ccun5uQkKDrrrtO06dPlyQtW7ZM8+bN04EDB9S6dWvde++9uuuuu6r8axmICE4AAAAIWIXFpeo8faNfPnvXrGRF2av3dfnzzz/Xli1b1KpVK+eyl19+WcXFxefMLEnS7bffrocfflgvvPCCEhMTtWrVKtWrV6/CsHHBBReUu/yll16Sw+HQgw8+6NZ2FcnMzFR0dLTeeecd57I5c+boq6++Utu2bSVJ//d//6dPP/1Ur7zyiiRp1apVmj59up5++mldcskl2rFjh8aOHau6detq9OjRbn1+ICE4AQAAAB7w5ptvql69eiopKVFRUZFCQkL09NNPO9fv3btXMTExatas2Tnb2u12XXjhhdq7d68k6csvv9SFF17o9gNbv/zyS0VHR5f7GdVRt25dLVu2zOW5WgkJCVq9erWmTZsm6ZeglJiYqHbt2kmS0tLSNG/ePF177bWSpDZt2mjXrl1asmQJwQkAAADwhsiwUO2aley3z3bHwIEDtXjxYhUUFGjBggWqU6eOrrvuump9tjGm2tt58hlYXbt2PedhxCNHjtTy5cs1bdo0GWP0wgsvKDU1VZJUUFCgr776SrfeeqvGjh3r3KakpEQxMTEeq8sfCE4AAAAIWDabrdqXy/la3bp1nbMuy5cvV0JCgp599lndeuutkqT27dsrLy9P3333neLi4ly2dTgc+uqrrzRw4EDn2A8//FDFxcVuzTqd+YwjR45UOusUEhJyTjgrLi4u95jONnz4cD300EP65JNPVFhYqMOHD2vYsGGSpJ9//lmStHTpUiUmJrpsFxrqXhANNHTVAwAAADwsJCREDz/8sKZOnarCwkJJ0nXXXaewsLByO9tlZGSooKBAw4cPlySNGDFCP//8sxYtWlTu/itqDpGSkiK73a65c+dWul3jxo2Vm5vrEp5+2yiiMi1atNCAAQO0atUqrVq1SldeeaWaNGkiSYqNjVVcXJz279+vdu3aubzatGlTpf0HquCI7wAAAECQuf766zVx4kSlp6frgQceUMuWLTV37lxNmDBBERERGjVqlMLCwvQ///M/evjhhzVhwgTnLE1iYqIefPBB57OXrrnmGsXFxWnfvn3KyMjQH/7wB5due2fEx8drwYIFuvvuu5Wfn68bb7xRrVu31jfffKN//etfqlevnubNm6c//vGP+uGHHzR37lylpKRow4YNeuuttxQdHV2lYxs5cqTS0tLkcDhcuvdJ0syZM3XvvfcqJiZGgwYNUlFRkbZv367jx487L+kLRsw4AQAAAF5Qp04d3X333Zo7d64KCgokSePHj9drr72mDz74QL169dLFF1+s1atXa/HixXriiSdctn/88ce1evVqffzxx0pOTlaXLl2Umpqqbt26Vdpk4a677tLbb7/tDFwdO3bUmDFjFB0d7ezo16lTJy1atEjp6elKSEjQ1q1by+32V5GUlBT99NNPOnXqlIYOHeqybsyYMVq2bJlWrFihrl27asCAAVq5cmXQzzjZTHXvPAtS+fn5iomJUV5eXpUTtbeccpQ422ueT7tLAACAmuL06dM6cOCA2rRpo4iICH+XgxqgsnPKnWzAjBMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAFghMAAAAAWCA4AQAAAIAF+l8DAACgRtm5c6eWLFminJwcnTx5UvXr11f37t11++23KyEhwd/lIUgRnAAAAFAjbNu2TePHj9eWLVvOWZeVlaXFixerf//+WrBggXr37u2HChHMuFQPAAAAQW/dunW67LLLzglNZz/wdPPmzbrsssu0bt06X5aHGoDgBAAAgKC2bds2paSk6PTp05KkDh06aNGiRcrLy1NhYaFOnDihRYsWqUOHDpKk06dPKyUlRdu2bfNqXZs2bZLNZtOJEye8+jlnW7lypS644ILz2sfBgwdls9mUk5NT4Rh/HZ+/BERwSk9PV+vWrRUREaHExERt3bq10vEvvfSSOnbsqIiICHXt2lXr16/3UaUAAAAINOPHj3eGphtuuEGffvqp7rzzTkVHR0uSYmJidOedd2rnzp26/vrrJf0Snu6//36P1fDHP/5R48eP99j+EHj8HpzWrl2r1NRUpaWl6ZNPPlFCQoKSk5P1/ffflzt+y5YtGj58uG699Vbt2LFDQ4cO1dChQ/X555/7uHIAAAD4W05OjvPyvA4dOuj555+X3W4vd2x4eLj+/e9/O2eeNm/erJ07d/qs1qpwOBz+LiGgFRcX++2z/R6c5s+fr7Fjx+rmm29W586dlZGRoaioKC1fvrzc8U8++aQGDRqkiRMnqlOnTnrkkUfUo0cPPf300z6uHAAAAP72zDPPOH++7777KgxNZ9jtdt17773lbl9dN910k/773//qySeflM1mk81m08GDB53rs7Oz1atXL0VFRalfv37as2ePc92MGTPUvXt3LVu2TG3atHHek3XixAmNGTNGjRs3VnR0tC6//HKXkLdz504NHDhQ9evXV3R0tHr27Knt27e71LVx40Z16tRJ9erV06BBg3TkyBHnurKyMs2aNUstWrRQeHi4unfvrg0bNlR6nOvXr1f79u0VGRmpgQMHuhxjeVauXOn89fjta8aMGc4xy5YtU6dOnRQREaGOHTtq0aJFznVnLhdcu3atBgwYoIiICK1atapatXuCX4OTw+FQdna2kpKSnMtCQkKUlJSkrKyscrfJyspyGS9JycnJFY4vKipSfn6+ywsAAAA1w2/vwRkxYkSVthk5cmS521fXk08+qb59+2rs2LE6cuSIjhw5ovj4eOf6KVOmaN68edq+fbvq1KmjW265xWX7ffv26ZVXXtGrr77qrOf666/X999/r7feekvZ2dnq0aOHrrjiCh07dsx5DC1atNC2bduUnZ2tSZMmKSwszLnPU6dO6YknntDzzz+v999/X4cOHdIDDzzgUvO8efP0xBNP6NNPP1VycrL+8pe/6Msvvyz3GA8fPqxrr71WQ4YMUU5OjsaMGaNJkyZV+usybNgw56/HkSNH9MILL6hOnTrq37+/JGnVqlWaPn26Hn30Ue3evVuzZ8/WtGnT9Nxzz7nsZ9KkSbrvvvu0e/duJScnu127xxg/+vbbb40ks2XLFpflEydONH369Cl3m7CwMLN69WqXZenp6aZJkybljk9LSzOSznnl5eV55iDOQ0FRsWn10Jum1UNvmoKiYn+XAwAA4HeFhYVm165dprCwsErjL774YiPJREREuPU54eHhRpLp2rVrdco8x4ABA8x9993nsuy9994zksx//vMf57J169YZSc7jS0tLM2FhYeb77793jvnggw9MdHS0OX36tMv+2rZta5YsWWKMMaZ+/fpm5cqV5dayYsUKI8ns27fPuSw9Pd3ExsY638fFxZlHH33UZbvevXubu+66yxhjzIEDB4wks2PHDmOMMZMnTzadO3d2Gf/QQw8ZSeb48eMV/bI47du3zzRs2NDMnTvX5XjO/l7/yCOPmL59+7rUsHDhQpcxVrWfrbJzKi8vr8rZoMY/x2ny5MlKTU11vs/Pz3f5HwB/igwL1a5Zyc6fAQAA4J769etL+qXZQ15enmJiYiy3ycvLU1FRkcv23tStWzfnz82aNZMkff/992rZsqUkqVWrVmrcuLFzzM6dO/Xzzz/rd7/7nct+CgsL9dVXX0mSUlNTNWbMGD3//PNKSkrS9ddfr7Zt2zrHRkVFubxv1qyZs4dAfn6+vvvuO+fMzxn9+/ev8J6v3bt3KzEx0WVZ3759q3T8eXl5+vOf/6zBgwdr4sSJkqSCggJ99dVXuvXWWzV27Fjn2JKSknN+D3v16uX8uTq1e4pfg1OjRo0UGhqqo0ePuiw/evSomjZtWu42TZs2dWt8eHi4wsPDPVOwh9lsNkXZa3x2BQAA8Jru3bs7b9lYvXq17rzzTsttVq1a5bK9t/32EjqbzSbpl3uMzqhbt67L+J9//lnNmjXTpk2bztnXmTbjM2bM0IgRI7Ru3Tq99dZbSktL05o1a3TNNdec85lnPtcY44nDcUtpaamGDRum6Ohol/vJfv75Z0nS0qVLzwlkoaGuEwpn//r4i1/vcbLb7erZs6cyMzOdy8rKypSZmVlhgu3bt6/LeEl65513qpx4AQAAUHPcdtttzp+ffPJJy650RUVF+uc//1nu9ufDbrertLTUI/vq0aOHcnNzVadOHbVr187l1ahRI+e49u3b6/7779fbb7+ta6+9VitWrKjS/qOjoxUXF6fNmze7LN+8ebM6d+5c7jadOnU655FBH330keVn3X///frss8/0+uuvuzyMODY2VnFxcdq/f/85x9imTRuP1u4pfp/uSE1N1ejRo9WrVy/16dNHCxcuVEFBgW6++WZJ0o033qjmzZtrzpw5kn7pljJgwADNmzdPgwcP1po1a7R9+3aPdEQBAABAcOnevbv69eunLVu2aM+ePRo1alSFLcmLioo0atQoZ1e7/v37KyEhwSN1tG7dWh9//LEOHjyoevXqqWHDhtXeV1JSkvr27auhQ4dq7ty5at++vb777jutW7dO11xzjbp06aKJEycqJSVFbdq00TfffKNt27bpuuuuq/JnTJw4UWlpaWrbtq26d++uFStWKCcnx2U27rfuuOMOzZs3TxMnTtSYMWOUnZ2tlStXVvoZK1as0KJFi/Taa6/JZrMpNzdXklSvXj3Vq1dPM2fO1L333quYmBgNGjRIRUVF2r59u44fP+5yq8351u4xlndB+cBTTz1lWrZsaex2u+nTp4/56KOPnOsGDBhgRo8e7TL+xRdfNO3btzd2u9106dLFrFu3rsqf5c4NYAAAAPAtd5tDGGPM1q1bTUREhLMJWIcOHUx6ero5ceKEMcaYEydOmPT0dNOhQwfnmMjISLN161aP1b1nzx7z+9//3kRGRhpJ5sCBA87mEL9tnrBjxw7nemN+aQ6RkJBwzv7y8/PNPffcY+Li4kxYWJiJj483I0eONIcOHTJFRUXmb3/7m4mPjzd2u93ExcWZu+++2/lrtmLFChMTE+Oyv9dee8389qt/aWmpmTFjhmnevLkJCwszCQkJ5q233nKuP7s5hDHG/O///q9p166dCQ8PN5deeqlZvnx5pc0hRo8eXW6TtrS0NOeYVatWme7duxu73W4aNGhgLrvsMvPqq69WWENVaj+bp5pD2Izxw8WOfpSfn6+YmBjl5eU5nyYNAACAwHD69GkdOHDA5ZlGVbFu3TqlpKTo9OnTLsvDw8OdjSDOiIyM1EsvvaTBgwd7pGYEtsrOKXeygd8fgAsAAACcr8GDB+v9998/p9va2aGpf//++u9//0togtv8fo8TAAAA4Am9e/fWhx9+qJ07d+qZZ55RTk6OTp48qfr166t79+667bbbPHZPE2ofghMAAABqlISEBKWnp/u7DNQwXKoHAAAAABYITgAAAAg4tax/GbzIU+cSwQkAAAABIzQ0VJIsH2QLVNWZc+nMuVVd3OMEAACAgFGnTh1FRUXphx9+UFhYmEJC+H9+VF9ZWZl++OEHRUVFqU6d84s+BCcAAAAEDJvNpmbNmunAgQP6+uuv/V0OaoCQkBC1bNlSNpvtvPZDcAIAAEBAsdvtuuiii7hcDx5ht9s9MnNJcAIAAEDACQkJUUREhL/LAJy4aBQAAAAALBCcAAAAAMACwQkAAAAALNS6e5zOPAArPz/fz5UAAAAA8KczmaAqD8mtdcHp5MmTkqT4+Hg/VwIAAAAgEJw8eVIxMTGVjrGZqsSrGqSsrEzfffed6tevf9693D0hPz9f8fHxOnz4sKKjo/1dDgIc5wvcxTkDd3HOwF2cM3BXIJ0zxhidPHlScXFxli3La92MU0hIiFq0aOHvMs4RHR3t9xMHwYPzBe7inIG7OGfgLs4ZuCtQzhmrmaYzaA4BAAAAABYITgAAAABggeDkZ+Hh4UpLS1N4eLi/S0EQ4HyBuzhn4C7OGbiLcwbuCtZzptY1hwAAAAAAdzHjBAAAAAAWCE4AAAAAYIHgBAAAAAAWCE4AAAAAYIHg5GXp6elq3bq1IiIilJiYqK1bt1Y6/qWXXlLHjh0VERGhrl27av369T6qFIHCnXNm6dKluvTSS9WgQQM1aNBASUlJlucYah53/545Y82aNbLZbBo6dKh3C0TAcfecOXHihMaNG6dmzZopPDxc7du359+nWsbdc2bhwoXq0KGDIiMjFR8fr/vvv1+nT5/2UbXwt/fff19DhgxRXFycbDabXn/9dcttNm3apB49eig8PFzt2rXTypUrvV6nuwhOXrR27VqlpqYqLS1Nn3zyiRISEpScnKzvv/++3PFbtmzR8OHDdeutt2rHjh0aOnSohg4dqs8//9zHlcNf3D1nNm3apOHDh+u9995TVlaW4uPjddVVV+nbb7/1ceXwF3fPmTMOHjyoBx54QJdeeqmPKkWgcPeccTgcuvLKK3Xw4EG9/PLL2rNnj5YuXarmzZv7uHL4i7vnzOrVqzVp0iSlpaVp9+7devbZZ7V27Vo9/PDDPq4c/lJQUKCEhASlp6dXafyBAwc0ePBgDRw4UDk5ORo/frzGjBmjjRs3erlSNxl4TZ8+fcy4ceOc70tLS01cXJyZM2dOueNvuOEGM3jwYJdliYmJ5vbbb/dqnQgc7p4zZyspKTH169c3zz33nLdKRICpzjlTUlJi+vXrZ5YtW2ZGjx5t/vrXv/qgUgQKd8+ZxYsXmwsvvNA4HA5flYgA4+45M27cOHP55Ze7LEtNTTX9+/f3ap0ITJLMa6+9VumYBx980HTp0sVl2bBhw0xycrIXK3MfM05e4nA4lJ2draSkJOeykJAQJSUlKSsrq9xtsrKyXMZLUnJycoXjUbNU55w526lTp1RcXKyGDRt6q0wEkOqeM7NmzVKTJk106623+qJMBJDqnDNvvPGG+vbtq3Hjxik2NlYXX3yxZs+erdLSUl+VDT+qzjnTr18/ZWdnOy/n279/v9avX6+rr77aJzUj+ATLd+A6/i6gpvrxxx9VWlqq2NhYl+WxsbH64osvyt0mNze33PG5ubleqxOBozrnzNkeeughxcXFnfOXD2qm6pwzH374oZ599lnl5OT4oEIEmuqcM/v379e7776rkSNHav369dq3b5/uuusuFRcXKy0tzRdlw4+qc86MGDFCP/74o/7whz/IGKOSkhLdcccdXKqHClX0HTg/P1+FhYWKjIz0U2WumHECaojHHntMa9as0WuvvaaIiAh/l4MAdPLkSY0aNUpLly5Vo0aN/F0OgkRZWZmaNGmiZ555Rj179tSwYcM0ZcoUZWRk+Ls0BKhNmzZp9uzZWrRokT755BO9+uqrWrdunR555BF/lwacF2acvKRRo0YKDQ3V0aNHXZYfPXpUTZs2LXebpk2bujUeNUt1zpkznnjiCT322GP6z3/+o27dunmzTAQQd8+Zr776SgcPHtSQIUOcy8rKyiRJderU0Z49e9S2bVvvFg2/qs7fM82aNVNYWJhCQ0Odyzp16qTc3Fw5HA7Z7Xav1gz/qs45M23aNI0aNUpjxoyRJHXt2lUFBQW67bbbNGXKFIWE8P/2cFXRd+Do6OiAmW2SmHHyGrvdrp49eyozM9O5rKysTJmZmerbt2+52/Tt29dlvCS98847FY5HzVKdc0aS5s6dq0ceeUQbNmxQr169fFEqAoS750zHjh312WefKScnx/n6y1/+4uxiFB8f78vy4QfV+Xumf//+2rdvnzNkS9LevXvVrFkzQlMtUJ1z5tSpU+eEozPB2xjjvWIRtILmO7C/u1PUZGvWrDHh4eFm5cqVZteuXea2224zF1xwgcnNzTXGGDNq1CgzadIk5/jNmzebOnXqmCeeeMLs3r3bpKWlmbCwMPPZZ5/56xDgY+6eM4899pix2+3m5ZdfNkeOHHG+Tp486a9DgI+5e86cja56tY+758yhQ4dM/fr1zd1332327Nlj3nzzTdOkSRPz97//3V+HAB9z95xJS0sz9evXNy+88ILZv3+/efvtt03btm3NDTfc4K9DgI+dPHnS7Nixw+zYscNIMvPnzzc7duwwX3/9tTHGmEmTJplRo0Y5x+/fv99ERUWZiRMnmt27d5v09HQTGhpqNmzY4K9DKBfBycueeuop07JlS2O3202fPn3MRx995Fw3YMAAM3r0aJfxL774omnfvr2x2+2mS5cuZt26dT6uGP7mzjnTqlUrI+mcV1pamu8Lh9+4+/fMbxGcaid3z5ktW7aYxMREEx4ebi688ELz6KOPmpKSEh9XDX9y55wpLi42M2bMMG3btjUREREmPj7e3HXXXeb48eO+Lxx+8d5775X7/eTMeTJ69GgzYMCAc7bp3r27sdvt5sILLzQrVqzwed1WbMYwZwoAAAAAleEeJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwAAAACwQHACAAAAAAsEJwBAjXHTTTfJZrOd89q3b5/LOrvdrnbt2mnWrFkqKSmRJG3atMllm8aNG+vqq6/WZ5995uejAgAEAoITAKBGGTRokI4cOeLyatOmjcu6L7/8UhMmTNCMGTP0j3/8w2X7PXv26MiRI9q4caOKioo0ePBgORwOfxwKACCAEJwAADVKeHi4mjZt6vIKDQ11WdeqVSvdeeedSkpK0htvvOGyfZMmTdS0aVP16NFD48eP1+HDh/XFF1/441AAAAGE4AQAqLUiIyMrnE3Ky8vTmjVrJEl2u92XZQEAAlAdfxcAAIAnvfnmm6pXr57z/Z/+9Ce99NJLLmOMMcrMzNTGjRt1zz33uKxr0aKFJKmgoECS9Je//EUdO3b0ctUAgEBHcAIA1CgDBw7U4sWLne/r1q3r/PlMqCouLlZZWZlGjBihGTNmuGz/wQcfKCoqSh999JFmz56tjIwMX5UOAAhgBCcAQI1St25dtWvXrtx1Z0KV3W5XXFyc6tQ595/BNm3a6IILLlCHDh30/fffa9iwYXr//fe9XTYAIMBxjxMAoNY4E6patmxZbmg627hx4/T555/rtdde80F1AIBARnACAKACUVFRGjt2rNLS0mSM8Xc5AAA/IjgBAFCJu+++W7t37z6nwQQAoHaxGf4LDQAAAAAqxYwTAAAAAFggOAEAAACABYITAAAAAFggOAEAAACABYITAAAAAFggOAEAAACABYITAAAAAFggOAEAAACABYITAAAAAFggOAEAAACABYITAAAAAFggOAEAAACAhf8PtDD5LoyvtxcAAAAASUVORK5CYII=\n"},"metadata":{}}],"source":["#写入代码\n","fpr,tpr,thresholds = roc_curve(y_test, lr.decision_function(X_test))\n","plt.plot(fpr,tpr,label=\"ROC Curve\")\n","plt.xlabel(\"FPR\")\n","plt.ylabel(\"TPR(recall)\")\n","#找到最接近于0的阈值\n","close_zero = np.argmin(np.abs(thresholds))\n","plt.plot(fpr[close_zero],tpr[close_zero],'o',markersize=10,label=\"threshold zero\",fillstyle=\"none\",c='k',mew=2)\n","plt.legend(loc=4)"]},{"cell_type":"markdown","metadata":{"id":"xvkdVNTycQft"},"source":["#### 思考6\n","* 对于多分类问题如何绘制ROC曲线"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"PxjNm8wKcQft"},"outputs":[],"source":["#思考回答\n"]},{"cell_type":"markdown","metadata":{"id":"MrT89c-QcQft"},"source":["【思考】你能从这条OCR曲线的到什么信息？这些信息可以做什么？"]},{"cell_type":"code","execution_count":null,"metadata":{"id":"wOweld5ncQfu"},"outputs":[],"source":["#思考回答\n"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.12"},"toc":{"base_numbering":1,"nav_menu":{},"number_sections":true,"sideBar":true,"skip_h1_title":false,"title_cell":"Table of Contents","title_sidebar":"Contents","toc_cell":false,"toc_position":{"height":"calc(100% - 180px)","left":"10px","top":"150px","width":"384px"},"toc_section_display":true,"toc_window_display":true},"colab":{"provenance":[]}},"nbformat":4,"nbformat_minor":0}