We have acquired our data, performed exploratory data analysis, created charts for visualizations, created map visualizations, and feature engineered our data. Now it is time to utilize machine learning to create predictive models.
The first order of business is to import the libraries we are going to use.
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, f1_score
From then we set our X and y values. We will be predicting a positive or negative outcome based on the features of our data. Therefore we set y to resolution_outcome which will either be 0 (negative) or 1 (positive). X will be all the features of our data, except for resolution_outcome so we set X to everything except resolution_outcome by dropping it.
X = aug.drop('resolution_outcome', axis=1)
y = aug['resolution_outcome']
y_labels = list(y.unique())
feature_cols = X.columns
We then use the sklearn library to split our data into a training set and a testing set. using train_test_split. We previously set X to all our features except for resolution_outcome, here we take a portion of our X data and assign it to X_train, then we take the remainder of X and assign it to X_test. We do the same with y and assign a portion of y as y_train, then assign the remaining y data as y_test.
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=3)
Our first predictive model we will use is Logistic Regression. We use logistic regression because what we are predicting is categorical (positive outcome vs negative outcome) and is not continuous. We .fit our training data of X_train and y_train. Fitting your model to the training data is essentially the training part of the modeling process. It finds the coefficients for the equation specified via the algorithm being used.
logreg = LogisticRegression(fit_intercept = False, solver='liblinear')
model_log = logreg.fit(X_train, y_train)
Now that we have trained our model from the training set, we can use this model to predict on our test data using .predict
y_hat_test = logreg.predict(X_test)
y_hat_train = logreg.predict(X_train)
residuals = np.abs(y_train - y_hat_train)
print(pd.Series(residuals).value_counts())
print(pd.Series(residuals).value_counts(normalize=True))
In logistic regression, the residuals can be defined as observed minus expected values. This will give us an idea of how well our model is predicting. However, let’s use a different metric to gauge how well our model is predicting. A F1 Score is an ideal metric to use in this situation. Further explanation can be found here.
logreg = LogisticRegression(solver='liblinear')
logrreg.fit(X_train,y_train)
logreg_pred = logreg.predict(X_test)
# checking accuracy
print('Test Accuracy score: ', accuracy_score(y_test, lr_pred))
# checking F1 Score
print('Test F1 score: ', f1_score(y_test, lr_pred))
Our initial model gives us the following:
Test Accuracy score: 0.748 Test F1 score: 0.5935483870967742
Now, we have made a predictive model and used F1 as our metric to determine how well our model predicts. Reminded: the closer our F1 score is to 1, the better. Is that it? Are we done? No. Using SKLearn’s library for logistic regression, since we did not specify specific parameters, we used the default parameters. Looking at SKLearn’s documentation, there are 15 different parameters we can tweak and change to find a more optimal F1 Score.
In fact, there are many different things we can do to find our optimal F1 Score. More of that next week…
Fhel's Blog 