utils#

Utility methods for EvalML objectives.

Module Contents#

Functions#

get_all_objective_names

Get a list of the names of all objectives.

get_core_objective_names

Get a list of all valid core objectives.

get_core_objectives

Returns all core objective instances associated with the given problem type.

get_non_core_objectives

Get non-core objective classes.

get_objective

Returns the Objective class corresponding to a given objective name.

get_optimization_objectives

Get objectives for optimization.

get_ranking_objectives

Get objectives for pipeline rankings.

ranking_only_objectives

Get ranking-only objective classes.

Contents#

evalml.objectives.utils.get_all_objective_names()[source]#

Get a list of the names of all objectives.

Returns

Objective names

Return type

list (str)

evalml.objectives.utils.get_core_objective_names()[source]#

Get a list of all valid core objectives.

Returns

Objective names.

Return type

list[str]

evalml.objectives.utils.get_core_objectives(problem_type)[source]#

Returns all core objective instances associated with the given problem type.

Core objectives are designed to work out-of-the-box for any dataset.

Parameters

problem_type (str/ProblemTypes) – Type of problem

Returns

List of ObjectiveBase instances

Examples

>>> for objective in get_core_objectives("regression"):
...     print(objective.name)
ExpVariance
MaxError
MedianAE
MSE
MAE
R2
Root Mean Squared Error
>>> for objective in get_core_objectives("binary"):
...     print(objective.name)
MCC Binary
Log Loss Binary
Gini
AUC
Precision
F1
Balanced Accuracy Binary
Accuracy Binary
evalml.objectives.utils.get_non_core_objectives()[source]#

Get non-core objective classes.

Non-core objectives are objectives that are domain-specific. Users typically need to configure these objectives before using them in AutoMLSearch.

Returns

List of ObjectiveBase classes

evalml.objectives.utils.get_objective(objective, return_instance=False, **kwargs)[source]#

Returns the Objective class corresponding to a given objective name.

Parameters
  • objective (str or ObjectiveBase) – Name or instance of the objective class.

  • return_instance (bool) – Whether to return an instance of the objective. This only applies if objective is of type str. Note that the instance will be initialized with default arguments.

  • kwargs (Any) – Any keyword arguments to pass into the objective. Only used when return_instance=True.

Returns

ObjectiveBase if the parameter objective is of type ObjectiveBase. If objective is instead a valid objective name, function will return the class corresponding to that name. If return_instance is True, an instance of that objective will be returned.

Raises
  • TypeError – If objective is None.

  • TypeError – If objective is not a string and not an instance of ObjectiveBase.

  • ObjectiveNotFoundError – If input objective is not a valid objective.

  • ObjectiveCreationError – If objective cannot be created properly.

evalml.objectives.utils.get_optimization_objectives(problem_type)[source]#

Get objectives for optimization.

Parameters

problem_type (str/ProblemTypes) – Type of problem

Returns

List of ObjectiveBase instances

evalml.objectives.utils.get_ranking_objectives(problem_type)[source]#

Get objectives for pipeline rankings.

Parameters

problem_type (str/ProblemTypes) – Type of problem

Returns

List of ObjectiveBase instances

evalml.objectives.utils.ranking_only_objectives()[source]#

Get ranking-only objective classes.

Ranking-only objectives are objectives that are useful for evaluating the performance of a model, but should not be used as an optimization objective during AutoMLSearch for various reasons.

Returns

List of ObjectiveBase classes