Scikit-learn Evaluation Metrics: A Complete Guide to Measuring Machine Learning Models
Scikit-learn's Evaluation Metrics For Classification
Evaluating a machine learning model is an important step in understanding how well it performs and whether its predictions are reliable. Scikit-learn provides a range of evaluation metrics that can be used to assess different types of models. For classification tasks, metrics such as accuracy, precision, recall, and F1 score help measure prediction performance, while the confusion matrix and classification report provide additional details. For regression tasks, metrics such as Mean Squared Error (MSE) and Mean Absolute Error (MAE) measure how closely the model's predictions match the actual values. In this guide, we'll explore these Scikit-learn evaluation metrics, explain what they measure, and show how they can be used to evaluate machine learning models.
TP = True Positive, TN = True Negative, FP = False Positive, FN = False Negative
You can use the sklearn.metrics module to measure the performance of your module. The sklearn.metrics module offer different metrics to evaluate a classification model. You can check the full list of scikit-learn metrics.
Accuracy Score
The accuracy score is one of the most important metrics for classification models to measure the accuracy of the model.
Accuracy Score = (TP + TN) / (TP + TN + FP + FN)
from sklearn.metrics import accuracy_score
accuracy_score(y_test, y_pred)
For a practical example, see the example in the Scikit-learn course. The best performance is 1.
Recall Score
Scikit-learn's another evaluation metric for classification is recall.
Recall = TP / (TP + FN)
from sklearn.metrics import recall_score
recall_score(y_test, y_pred)
The best performance is 1, and the worst performance is 0.
Precision score
You can also use precision to measure the performance of your model.
Precision = TP / (TP + FP)
from sklearn.metrics import precision_score
precision_score(y_test, y_pred)
The best performance of precision is 1, and the worst performance is 0.
F1 Score
The F1 score (F-score or F-measure) combines both precision and recall into a single statistic.
F1-Score = (2 x precision x recall) / (precision + recall)
from sklearn.metrics import f1_score
f1_score(y_test, y_pred)
Classification Report
The classification_report is used to build a text report showing the main classification metrics.
from sklearn.metrics import classification_report
classification_report(y_true, y_pred, target_names=target_names)
Using target_names parameter is optional.
You can read the classification report of the logistic regression model that we created.
A classification report displays precision, recall, f1score, accuracy, and support scores of a model. Support refers to the number of actual occurrences of the class in the dataset. You can find a practical example of this in the Scikit-learn course.
Confusion Matrix
The confusion matrix computes confusion matrix to evaluate the accuracy of a classification.
from sklearn.metrics import confusion_matrix
print(confusion_matrix(y_true, y_pred))
Confusion matrix can be difficult to understand. You can learn how to interpret a confusion matrix with an example.
Scikit-learn's Evaluation Metrics for Regression
The sklearn.metrics module provides evaluation metrics for regression models as well. Two commonly used metrics are Mean Squared Error (MSE) and Mean Absolute Error (MAE). Both metrics measure the difference between the actual and predicted values, but they calculate the error differently.
mean_squared_error is a non-negative floating point value and the best score is 0.0.
mean_absolute_error is a non-negative floating point value and the best score is 0.0.
You can find examples of mean_squared_error and mean_absolute_error in our Scikit-learn course.
Another important evaluation metric is r2_score. r2_score is the regression score function and the best value is 1.0. It can be negative as well.
from sklearn.metrics import mean_squared_error
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import r2_score
mse = mean_squared_error(y_test, y_pred)
mae = mean_absolute_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)
We imported mean-squared_error, mean_absolute_error, and r2_score separately to show how to import the metrics, but you can import them together: from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score. If you want to learn other metrics for regression models, read the scikit-learn's metrics page.