Hyperparameters for the Support Vector Machines :Choose the Best

Support Vector Machine is one of the popular machine learning algorithms. If you have earlier build the machine learning model using a support vector machine, then this tutorial is for you. You will learn how to optimize your model accuracy using the SVM() parameters. In this intuition, you will know how to find the best hyperparameters for the Support Vector Machines

Support Vector Machine Basics

SVM is a classifier that finds the optimal hyperplane to maximize the margin between the two classes. I am not going into details about how hyperplane is calculated. You can read here to know more about Hyperplane formulae.
Linearly separable data is easily classified using the SVM, but you have to use the kernel trick for classifying the non linearly separable data. Kernel transform n-dimensional data to higher dimension data. For example from the 2 -Dimension data to 3-Dimension data.

When to apply the Support Vector Machine?

The followings things you keep in mind before applying support Vector Machine.

1. The target variable should be binary.
2. The ratio of the number of variables and rows should be high. It means that use it when you have a lot of features and few rows. It’s due that the SVM algorithm takes a long time to train.
3. If your dataset has a lot of outliers as SVM works on the points nearest to the line. Therefore outliers are ignored.

You cannot use the Support Vector Machine for a quick benchmark model. Use the simple algorithms for it.

Best Hyperparameters for the Support Vector Machine

For the best model accuracies let’s optimize the hyperparameters of the SVC by step by step.

Step 1: Import the Support vector classifier using the sklearn package

import numpy as np
import pandas as pd
from sklearn.svm import SVC

Step 2: Print out the SVC Hyperparameters.

print(SVC())

You can see there are various hyperparameters for the svc. But we will take the important one C and the kernel.

Why C is important?

The value of C determines the penalty for the classifier. If C is very large, then there will be a large penalty for misclassification in training, thus small margin. And if the C is small, then a small penalty and large margin.
A brief about Kernel has been already being defined in SVM basics.

Step 3: Use the GridSearchCv for finding the best parameters.

Sklearn module has a GridSearchCv cross-validation technique for finding the best parameters. Let’s define the parameters for your classifier SVC().

svc= SVC()
parameters = {
    "kernel": ["linear","rbf"],
    "C":[0.1,1,10]
}

Now you will use the GridSearchCV() method and pass the parameters and estimator or model into as the arguments. After that, you will fit it on the train features and train labels.

cv = GridSearchCV(svc,parameters,cv=5)
cv.fit(train_features,train_label.values.ravel())

Here inside the method, svc is the support vector classifier, cv is the number of the valid iterators. The default is 3 but here it is 5. It means it will train the model five times.

Download the complete dataset.

Step 4: Find the best parameters and display all the results.

You can easily find the best parameters using the cv.best_params_. Let’s print out the best score and parameters in a well-mannered way. Below is the display function that prints out the best parameters and all the scores for each iteration.

def display(results):
    print(f'Best parameters are: {results.best_params_}')
    mean_score = results.cv_results_['mean_test_score']
    std_score = results.cv_results_['std_test_score']
    params = results.cv_results_['params']
    for mean,std,params in zip(mean_score,std_score,params):
        print(f'{round(mean,2)} + or -{round(std,2)} for the {params}')

Passing the cv as the argument will display out the best parameter and scores. You will get the following output.

display(cv)

You can clearly see the GridSearchCV has shown the best the parameter for the model. It is 1 for C and linear kernel. Use it for improving the accuracy of the model in the given dataset.

Conclusion

There is various way you can improve the score of the machine learning model. Like training on the more data or tuning the hyperparameters for that model like in this case Hyperparameters for the Support Vector Machines. I suggest you tune the parameter of the model first, then move to add more data for training. This way you can increase the efficiency and reduce the cost of the data creation.