Hyperparameter tuning

Before going to this blog make sure you are aware about any one of the algorithms like Decision tree, Random forest.etc….Below are the few  parameters that we need for most of the algorithms.

1. Maximum number of leaves per tree
2. Depth of trees.
3. Number of trees in Random forest
4. Learning rate
5. L1 and L2 regularization weights.

We have to set perfect parameters (which is called Hyper tune parameters) to achieve the best accuracy,but it is very difficult to identify the best parameters, so there is a option in all Machine Learning packages to check the fine tuning parameters.
We will discuss about some of the properties that we need to set when we want to find the best hypertune parameter using Azure ML(This properties remains same in all Machine learning tools/packages).

The very important parameter is sweeping mode which has below three options.

1. Entire grid
2. Random grid
3. Random sweep

Before discussing about the above mentioned sweeping mode, First what is grid? It is a framework of spaced bars that are parallel to or cross each other.coming to our context it is cartesian product of parameters.

Let’s take an example for clear understanding.

Parameter 1 = 1,2,3

Parameter 2 = A, B, C, D

We have two parameters now our final grid will looks like below.This grid is the all possible combinations of Parameter 1 and Parameter 2

Hope you are clear what is grid in the context of hypertune parameters.Now we will discuss about our three types of sweep parameters.

Random Grid:

Using this option our algorithm will run on randomly selected combination parameters as shown below.

Entire Grid:

It will try with all possible combination  parameters.As algorithm will run with all possible combinations, it takes more time than Random Grid.

Random Sweep:

The module will randomly select parameter values over a system-defined range. You must specify the maximum number of runs that you want the module to execute. This option is useful for cases where you want to increase model performance using the metrics of your choice but still conserve computing resources.