Tuners#

EvalML tuner classes.

Package Contents#

Classes Summary#

GridSearchTuner

Grid Search Optimizer, which generates all of the possible points to search for using a grid.

RandomSearchTuner

Random Search Optimizer.

SKOptTuner

Bayesian Optimizer.

Tuner

Base Tuner class.

Exceptions Summary#

Contents#

class evalml.tuners.GridSearchTuner(pipeline_hyperparameter_ranges, n_points=10, random_seed=0)[source]#

Grid Search Optimizer, which generates all of the possible points to search for using a grid.

Parameters

  • pipeline_hyperparameter_ranges (dict) – a set of hyperparameter ranges corresponding to a pipeline’s parameters

  • n_points (int) – The number of points to sample from along each dimension defined in the space argument. Defaults to 10.

  • random_seed (int) – Seed for random number generator. Unused in this class, defaults to 0.

Examples

>>> tuner = GridSearchTuner({'My Component': {'param a': [0.0, 10.0], 'param b': ['a', 'b', 'c']}}, n_points=5)
>>> proposal = tuner.propose()
...
>>> assert proposal.keys() == {'My Component'}
>>> assert proposal['My Component'] == {'param a': 0.0, 'param b': 'a'}

Determines points using a grid search approach.

>>> for each in range(5):
...     print(tuner.propose())
{'My Component': {'param a': 0.0, 'param b': 'b'}}
{'My Component': {'param a': 0.0, 'param b': 'c'}}
{'My Component': {'param a': 10.0, 'param b': 'a'}}
{'My Component': {'param a': 10.0, 'param b': 'b'}}
{'My Component': {'param a': 10.0, 'param b': 'c'}}

Methods

add

Not applicable to grid search tuner as generated parameters are not dependent on scores of previous parameters.

get_starting_parameters

Gets the starting parameters given the pipeline hyperparameter range.

is_search_space_exhausted

Checks if it is possible to generate a set of valid parameters. Stores generated parameters in self.curr_params to be returned by propose().

propose

Returns parameters from _grid_points iterations.
add(self, pipeline_parameters, score)[source]#

Not applicable to grid search tuner as generated parameters are not dependent on scores of previous parameters.

Parameters

  • pipeline_parameters (dict) – a dict of the parameters used to evaluate a pipeline

  • score (float) – the score obtained by evaluating the pipeline with the provided parameters

get_starting_parameters(self, hyperparameter_ranges, random_seed=0)#

Gets the starting parameters given the pipeline hyperparameter range.

Parameters

  • hyperparameter_ranges (dict) – The custom hyperparameter ranges passed in during search. Used to determine the starting parameters.

  • random_seed (int) – The random seed to use. Defaults to 0.

Returns

The starting parameters, randomly chosen, to initialize a pipeline with.

Return type

dict

is_search_space_exhausted(self)[source]#

Checks if it is possible to generate a set of valid parameters. Stores generated parameters in self.curr_params to be returned by propose().

Returns

If no more valid parameters exists in the search space, return False.

Return type

bool

Raises

NoParamsException – If a search space is exhausted, then this exception is thrown.

propose(self)[source]#

Returns parameters from _grid_points iterations.

If all possible combinations of parameters have been scored, then NoParamsException is raised.

Returns

proposed pipeline parameters

Return type

dict

exception evalml.tuners.NoParamsException[source]#

Raised when a tuner exhausts its search space and runs out of parameters to propose.

exception evalml.tuners.ParameterError[source]#

Raised when a tuner encounters an error with the parameters being used with it.

class evalml.tuners.RandomSearchTuner(pipeline_hyperparameter_ranges, with_replacement=False, replacement_max_attempts=10, random_seed=0)[source]#

Random Search Optimizer.

Parameters

  • pipeline_hyperparameter_ranges (dict) – a set of hyperparameter ranges corresponding to a pipeline’s parameters

  • with_replacement (bool) – If false, only unique hyperparameters will be shown

  • replacement_max_attempts (int) – The maximum number of tries to get a unique set of random parameters. Only used if tuner is initalized with with_replacement=True

  • random_seed (int) – Seed for random number generator. Defaults to 0.

Example

>>> tuner = RandomSearchTuner({'My Component': {'param a': [0.0, 10.0], 'param b': ['a', 'b', 'c']}}, random_seed=42)
>>> proposal = tuner.propose()
...
>>> assert proposal.keys() == {'My Component'}
>>> assert proposal['My Component'] == {'param a': 3.7454011884736254, 'param b': 'c'}

Determines points using a random search approach.

>>> for each in range(7):
...     print(tuner.propose())
{'My Component': {'param a': 7.3199394181140525, 'param b': 'b'}}
{'My Component': {'param a': 1.5601864044243654, 'param b': 'a'}}
{'My Component': {'param a': 0.5808361216819947, 'param b': 'c'}}
{'My Component': {'param a': 6.011150117432089, 'param b': 'c'}}
{'My Component': {'param a': 0.2058449429580245, 'param b': 'c'}}
{'My Component': {'param a': 8.32442640800422, 'param b': 'a'}}
{'My Component': {'param a': 1.8182496720710064, 'param b': 'a'}}

Methods

add

Not applicable to random search tuner as generated parameters are not dependent on scores of previous parameters.

get_starting_parameters

Gets the starting parameters given the pipeline hyperparameter range.

is_search_space_exhausted

Checks if it is possible to generate a set of valid parameters. Stores generated parameters in self.curr_params to be returned by propose().

propose

Generate a unique set of parameters.
add(self, pipeline_parameters, score)[source]#

Not applicable to random search tuner as generated parameters are not dependent on scores of previous parameters.

Parameters

  • pipeline_parameters (dict) – A dict of the parameters used to evaluate a pipeline

  • score (float) – The score obtained by evaluating the pipeline with the provided parameters

get_starting_parameters(self, hyperparameter_ranges, random_seed=0)#

Gets the starting parameters given the pipeline hyperparameter range.

Parameters

  • hyperparameter_ranges (dict) – The custom hyperparameter ranges passed in during search. Used to determine the starting parameters.

  • random_seed (int) – The random seed to use. Defaults to 0.

Returns

The starting parameters, randomly chosen, to initialize a pipeline with.

Return type

dict

is_search_space_exhausted(self)[source]#

Checks if it is possible to generate a set of valid parameters. Stores generated parameters in self.curr_params to be returned by propose().

Returns

If no more valid parameters exists in the search space, return False.

Return type

bool

Raises

NoParamsException – If a search space is exhausted, then this exception is thrown.

propose(self)[source]#

Generate a unique set of parameters.

If tuner was initialized with with_replacement=True and the tuner is unable to generate a unique set of parameters after replacement_max_attempts tries, then NoParamsException is raised.

Returns

Proposed pipeline parameters

Return type

dict

class evalml.tuners.SKOptTuner(pipeline_hyperparameter_ranges, random_seed=0)[source]#

Bayesian Optimizer.

Parameters

  • pipeline_hyperparameter_ranges (dict) – A set of hyperparameter ranges corresponding to a pipeline’s parameters.

  • random_seed (int) – The seed for the random number generator. Defaults to 0.

Examples

>>> tuner = SKOptTuner({'My Component': {'param a': [0.0, 10.0], 'param b': ['a', 'b', 'c']}})
>>> proposal = tuner.propose()
...
>>> assert proposal.keys() == {'My Component'}
>>> assert proposal['My Component'] == {'param a': 5.928446182250184, 'param b': 'c'}

Determines points using a Bayesian Optimizer approach.

>>> for each in range(7):
...     print(tuner.propose())
{'My Component': {'param a': 8.57945617622757, 'param b': 'c'}}
{'My Component': {'param a': 6.235636967859724, 'param b': 'b'}}
{'My Component': {'param a': 2.9753460654447235, 'param b': 'a'}}
{'My Component': {'param a': 2.7265629458011325, 'param b': 'b'}}
{'My Component': {'param a': 8.121687287754932, 'param b': 'b'}}
{'My Component': {'param a': 3.927847961008298, 'param b': 'c'}}
{'My Component': {'param a': 3.3739616041726843, 'param b': 'b'}}

Methods

add

Add score to sample.

get_starting_parameters

Gets the starting parameters given the pipeline hyperparameter range.

is_search_space_exhausted

Optional. If possible search space for tuner is finite, this method indicates whether or not all possible parameters have been scored.

propose

Returns a suggested set of parameters to train and score a pipeline with, based off the search space dimensions and prior samples.
add(self, pipeline_parameters, score)[source]#

Add score to sample.

Parameters

  • pipeline_parameters (dict) – A dict of the parameters used to evaluate a pipeline

  • score (float) – The score obtained by evaluating the pipeline with the provided parameters

Returns

None

Raises

  • Exception – If skopt tuner errors.

  • ParameterError – If skopt receives invalid parameters.

get_starting_parameters(self, hyperparameter_ranges, random_seed=0)#

Gets the starting parameters given the pipeline hyperparameter range.

Parameters

  • hyperparameter_ranges (dict) – The custom hyperparameter ranges passed in during search. Used to determine the starting parameters.

  • random_seed (int) – The random seed to use. Defaults to 0.

Returns

The starting parameters, randomly chosen, to initialize a pipeline with.

Return type

dict

is_search_space_exhausted(self)#

Optional. If possible search space for tuner is finite, this method indicates whether or not all possible parameters have been scored.

Returns

Returns true if all possible parameters in a search space has been scored.

Return type

bool

propose(self)[source]#

Returns a suggested set of parameters to train and score a pipeline with, based off the search space dimensions and prior samples.

Returns

Proposed pipeline parameters.

Return type

dict

class evalml.tuners.Tuner(pipeline_hyperparameter_ranges, random_seed=0)[source]#

Base Tuner class.

Tuners implement different strategies for sampling from a search space. They’re used in EvalML to search the space of pipeline hyperparameters.

Parameters

  • pipeline_hyperparameter_ranges (dict) – a set of hyperparameter ranges corresponding to a pipeline’s parameters.

  • random_seed (int) – The random state. Defaults to 0.

Methods

add

Register a set of hyperparameters with the score obtained from training a pipeline with those hyperparameters.

get_starting_parameters

Gets the starting parameters given the pipeline hyperparameter range.

is_search_space_exhausted

Optional. If possible search space for tuner is finite, this method indicates whether or not all possible parameters have been scored.

propose

Returns a suggested set of parameters to train and score a pipeline with, based off the search space dimensions and prior samples.
abstract add(self, pipeline_parameters, score)[source]#

Register a set of hyperparameters with the score obtained from training a pipeline with those hyperparameters.

Parameters

  • pipeline_parameters (dict) – a dict of the parameters used to evaluate a pipeline

  • score (float) – the score obtained by evaluating the pipeline with the provided parameters

Returns

None

get_starting_parameters(self, hyperparameter_ranges, random_seed=0)[source]#

Gets the starting parameters given the pipeline hyperparameter range.

Parameters

  • hyperparameter_ranges (dict) – The custom hyperparameter ranges passed in during search. Used to determine the starting parameters.

  • random_seed (int) – The random seed to use. Defaults to 0.

Returns

The starting parameters, randomly chosen, to initialize a pipeline with.

Return type

dict

is_search_space_exhausted(self)[source]#

Optional. If possible search space for tuner is finite, this method indicates whether or not all possible parameters have been scored.

Returns

Returns true if all possible parameters in a search space has been scored.

Return type

bool

abstract propose(self)[source]#

Returns a suggested set of parameters to train and score a pipeline with, based off the search space dimensions and prior samples.

Returns

Proposed pipeline parameters

Return type

dict