standard_metrics ============================================ .. py:module:: evalml.objectives.standard_metrics .. autoapi-nested-parse:: Standard machine learning objective functions. Module Contents --------------- Classes Summary ~~~~~~~~~~~~~~~ .. autoapisummary:: evalml.objectives.standard_metrics.AccuracyBinary evalml.objectives.standard_metrics.AccuracyMulticlass evalml.objectives.standard_metrics.AUC evalml.objectives.standard_metrics.AUCMacro evalml.objectives.standard_metrics.AUCMicro evalml.objectives.standard_metrics.AUCWeighted evalml.objectives.standard_metrics.BalancedAccuracyBinary evalml.objectives.standard_metrics.BalancedAccuracyMulticlass evalml.objectives.standard_metrics.ExpVariance evalml.objectives.standard_metrics.F1 evalml.objectives.standard_metrics.F1Macro evalml.objectives.standard_metrics.F1Micro evalml.objectives.standard_metrics.F1Weighted evalml.objectives.standard_metrics.Gini evalml.objectives.standard_metrics.LogLossBinary evalml.objectives.standard_metrics.LogLossMulticlass evalml.objectives.standard_metrics.MAE evalml.objectives.standard_metrics.MAPE evalml.objectives.standard_metrics.MaxError evalml.objectives.standard_metrics.MCCBinary evalml.objectives.standard_metrics.MCCMulticlass evalml.objectives.standard_metrics.MeanSquaredLogError evalml.objectives.standard_metrics.MedianAE evalml.objectives.standard_metrics.MSE evalml.objectives.standard_metrics.Precision evalml.objectives.standard_metrics.PrecisionMacro evalml.objectives.standard_metrics.PrecisionMicro evalml.objectives.standard_metrics.PrecisionWeighted evalml.objectives.standard_metrics.R2 evalml.objectives.standard_metrics.Recall evalml.objectives.standard_metrics.RecallMacro evalml.objectives.standard_metrics.RecallMicro evalml.objectives.standard_metrics.RecallWeighted evalml.objectives.standard_metrics.RootMeanSquaredError evalml.objectives.standard_metrics.RootMeanSquaredLogError Contents ~~~~~~~~~~~~~~~~~~~ .. py:class:: AccuracyBinary Accuracy score for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(AccuracyBinary().objective_function(y_true, y_pred), 0.6363636) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Accuracy Binary * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.AccuracyBinary.calculate_percent_difference evalml.objectives.standard_metrics.AccuracyBinary.can_optimize_threshold evalml.objectives.standard_metrics.AccuracyBinary.decision_function evalml.objectives.standard_metrics.AccuracyBinary.is_defined_for_problem_type evalml.objectives.standard_metrics.AccuracyBinary.objective_function evalml.objectives.standard_metrics.AccuracyBinary.optimize_threshold evalml.objectives.standard_metrics.AccuracyBinary.positive_only evalml.objectives.standard_metrics.AccuracyBinary.score evalml.objectives.standard_metrics.AccuracyBinary.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for accuracy score for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: AccuracyMulticlass Accuracy score for multiclass classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(AccuracyMulticlass().objective_function(y_true, y_pred), 0.5454545) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Accuracy Multiclass * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.AccuracyMulticlass.calculate_percent_difference evalml.objectives.standard_metrics.AccuracyMulticlass.is_defined_for_problem_type evalml.objectives.standard_metrics.AccuracyMulticlass.objective_function evalml.objectives.standard_metrics.AccuracyMulticlass.positive_only evalml.objectives.standard_metrics.AccuracyMulticlass.score evalml.objectives.standard_metrics.AccuracyMulticlass.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for accuracy score for multiclass classification. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: AUC AUC score for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(AUC().objective_function(y_true, y_pred), 0.5714285) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - AUC * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - True **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.AUC.calculate_percent_difference evalml.objectives.standard_metrics.AUC.can_optimize_threshold evalml.objectives.standard_metrics.AUC.decision_function evalml.objectives.standard_metrics.AUC.is_defined_for_problem_type evalml.objectives.standard_metrics.AUC.objective_function evalml.objectives.standard_metrics.AUC.optimize_threshold evalml.objectives.standard_metrics.AUC.positive_only evalml.objectives.standard_metrics.AUC.score evalml.objectives.standard_metrics.AUC.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for AUC score for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: AUCMacro AUC score for multiclass classification using macro averaging. .. rubric:: Example >>> y_true = [0, 1, 2, 0, 2, 1] >>> y_pred = [[0.7, 0.2, 0.1], ... [0.1, 0.0, 0.9], ... [0.1, 0.3, 0.6], ... [0.9, 0.1, 0.0], ... [0.6, 0.1, 0.3], ... [0.5, 0.5, 0.0]] >>> np.testing.assert_almost_equal(AUCMacro().objective_function(y_true, y_pred), 0.75) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - AUC Macro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - True **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.AUCMacro.calculate_percent_difference evalml.objectives.standard_metrics.AUCMacro.is_defined_for_problem_type evalml.objectives.standard_metrics.AUCMacro.objective_function evalml.objectives.standard_metrics.AUCMacro.positive_only evalml.objectives.standard_metrics.AUCMacro.score evalml.objectives.standard_metrics.AUCMacro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for AUC score for multiclass classification using macro-averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: AUCMicro AUC score for multiclass classification using micro averaging. .. rubric:: Example >>> y_true = [0, 1, 2, 0, 2, 1] >>> y_pred = [[0.7, 0.2, 0.1], ... [0.3, 0.5, 0.2], ... [0.1, 0.3, 0.6], ... [0.9, 0.1, 0.0], ... [0.3, 0.1, 0.6], ... [0.5, 0.5, 0.0]] >>> np.testing.assert_almost_equal(AUCMicro().objective_function(y_true, y_pred), 0.9861111) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - AUC Micro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - True **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.AUCMicro.calculate_percent_difference evalml.objectives.standard_metrics.AUCMicro.is_defined_for_problem_type evalml.objectives.standard_metrics.AUCMicro.objective_function evalml.objectives.standard_metrics.AUCMicro.positive_only evalml.objectives.standard_metrics.AUCMicro.score evalml.objectives.standard_metrics.AUCMicro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for AUC score for multiclass classification using micro-averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: AUCWeighted AUC Score for multiclass classification using weighted averaging. .. rubric:: Example >>> y_true = [0, 1, 2, 0, 2, 1] >>> y_pred = [[0.7, 0.2, 0.1], ... [0.1, 0.0, 0.9], ... [0.1, 0.3, 0.6], ... [0.1, 0.2, 0.7], ... [0.6, 0.1, 0.3], ... [0.5, 0.2, 0.3]] >>> np.testing.assert_almost_equal(AUCWeighted().objective_function(y_true, y_pred), 0.4375) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - AUC Weighted * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - True **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.AUCWeighted.calculate_percent_difference evalml.objectives.standard_metrics.AUCWeighted.is_defined_for_problem_type evalml.objectives.standard_metrics.AUCWeighted.objective_function evalml.objectives.standard_metrics.AUCWeighted.positive_only evalml.objectives.standard_metrics.AUCWeighted.score evalml.objectives.standard_metrics.AUCWeighted.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for AUC Score for multiclass classification using weighted averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: BalancedAccuracyBinary Balanced accuracy score for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(BalancedAccuracyBinary().objective_function(y_true, y_pred), 0.60) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Balanced Accuracy Binary * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.BalancedAccuracyBinary.calculate_percent_difference evalml.objectives.standard_metrics.BalancedAccuracyBinary.can_optimize_threshold evalml.objectives.standard_metrics.BalancedAccuracyBinary.decision_function evalml.objectives.standard_metrics.BalancedAccuracyBinary.is_defined_for_problem_type evalml.objectives.standard_metrics.BalancedAccuracyBinary.objective_function evalml.objectives.standard_metrics.BalancedAccuracyBinary.optimize_threshold evalml.objectives.standard_metrics.BalancedAccuracyBinary.positive_only evalml.objectives.standard_metrics.BalancedAccuracyBinary.score evalml.objectives.standard_metrics.BalancedAccuracyBinary.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for accuracy score for balanced accuracy for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: BalancedAccuracyMulticlass Balanced accuracy score for multiclass classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(BalancedAccuracyMulticlass().objective_function(y_true, y_pred), 0.5555555) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Balanced Accuracy Multiclass * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.BalancedAccuracyMulticlass.calculate_percent_difference evalml.objectives.standard_metrics.BalancedAccuracyMulticlass.is_defined_for_problem_type evalml.objectives.standard_metrics.BalancedAccuracyMulticlass.objective_function evalml.objectives.standard_metrics.BalancedAccuracyMulticlass.positive_only evalml.objectives.standard_metrics.BalancedAccuracyMulticlass.score evalml.objectives.standard_metrics.BalancedAccuracyMulticlass.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for accuracy score for balanced accuracy for multiclass classification. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: ExpVariance Explained variance score for regression. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(ExpVariance().objective_function(y_true, y_pred), 0.7760736) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - True * - **is_bounded_like_percentage** - False * - **name** - ExpVariance * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.ExpVariance.calculate_percent_difference evalml.objectives.standard_metrics.ExpVariance.is_defined_for_problem_type evalml.objectives.standard_metrics.ExpVariance.objective_function evalml.objectives.standard_metrics.ExpVariance.positive_only evalml.objectives.standard_metrics.ExpVariance.score evalml.objectives.standard_metrics.ExpVariance.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for explained variance score for regression. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: F1 F1 score for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(F1().objective_function(y_true, y_pred), 0.25) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - F1 * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.F1.calculate_percent_difference evalml.objectives.standard_metrics.F1.can_optimize_threshold evalml.objectives.standard_metrics.F1.decision_function evalml.objectives.standard_metrics.F1.is_defined_for_problem_type evalml.objectives.standard_metrics.F1.objective_function evalml.objectives.standard_metrics.F1.optimize_threshold evalml.objectives.standard_metrics.F1.positive_only evalml.objectives.standard_metrics.F1.score evalml.objectives.standard_metrics.F1.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for F1 score for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: F1Macro F1 score for multiclass classification using macro averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(F1Macro().objective_function(y_true, y_pred), 0.5476190) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - F1 Macro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.F1Macro.calculate_percent_difference evalml.objectives.standard_metrics.F1Macro.is_defined_for_problem_type evalml.objectives.standard_metrics.F1Macro.objective_function evalml.objectives.standard_metrics.F1Macro.positive_only evalml.objectives.standard_metrics.F1Macro.score evalml.objectives.standard_metrics.F1Macro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for F1 score for multiclass classification using macro averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: F1Micro F1 score for multiclass classification using micro averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(F1Micro().objective_function(y_true, y_pred), 0.5454545) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - F1 Micro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.F1Micro.calculate_percent_difference evalml.objectives.standard_metrics.F1Micro.is_defined_for_problem_type evalml.objectives.standard_metrics.F1Micro.objective_function evalml.objectives.standard_metrics.F1Micro.positive_only evalml.objectives.standard_metrics.F1Micro.score evalml.objectives.standard_metrics.F1Micro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for F1 score for multiclass classification. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: F1Weighted F1 score for multiclass classification using weighted averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(F1Weighted().objective_function(y_true, y_pred), 0.5454545) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - F1 Weighted * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.F1Weighted.calculate_percent_difference evalml.objectives.standard_metrics.F1Weighted.is_defined_for_problem_type evalml.objectives.standard_metrics.F1Weighted.objective_function evalml.objectives.standard_metrics.F1Weighted.positive_only evalml.objectives.standard_metrics.F1Weighted.score evalml.objectives.standard_metrics.F1Weighted.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for F1 score for multiclass classification using weighted averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: Gini Gini coefficient for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(Gini().objective_function(y_true, y_pred), 0.1428571) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - True * - **is_bounded_like_percentage** - False * - **name** - Gini * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - True **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.Gini.calculate_percent_difference evalml.objectives.standard_metrics.Gini.can_optimize_threshold evalml.objectives.standard_metrics.Gini.decision_function evalml.objectives.standard_metrics.Gini.is_defined_for_problem_type evalml.objectives.standard_metrics.Gini.objective_function evalml.objectives.standard_metrics.Gini.optimize_threshold evalml.objectives.standard_metrics.Gini.positive_only evalml.objectives.standard_metrics.Gini.score evalml.objectives.standard_metrics.Gini.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for Gini coefficient for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: LogLossBinary Log Loss for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(LogLossBinary().objective_function(y_true, y_pred), 19.6601745) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - Log Loss Binary * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - True **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.LogLossBinary.calculate_percent_difference evalml.objectives.standard_metrics.LogLossBinary.can_optimize_threshold evalml.objectives.standard_metrics.LogLossBinary.decision_function evalml.objectives.standard_metrics.LogLossBinary.is_defined_for_problem_type evalml.objectives.standard_metrics.LogLossBinary.objective_function evalml.objectives.standard_metrics.LogLossBinary.optimize_threshold evalml.objectives.standard_metrics.LogLossBinary.positive_only evalml.objectives.standard_metrics.LogLossBinary.score evalml.objectives.standard_metrics.LogLossBinary.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for log loss for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: LogLossMulticlass Log Loss for multiclass classification. .. rubric:: Example >>> y_true = [0, 1, 2, 0, 2, 1] >>> y_pred = [[0.7, 0.2, 0.1], ... [0.3, 0.5, 0.2], ... [0.1, 0.3, 0.6], ... [0.9, 0.1, 0.0], ... [0.3, 0.1, 0.6], ... [0.5, 0.5, 0.0]] >>> np.testing.assert_almost_equal(LogLossMulticlass().objective_function(y_true, y_pred), 0.4783301) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - Log Loss Multiclass * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - True **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.LogLossMulticlass.calculate_percent_difference evalml.objectives.standard_metrics.LogLossMulticlass.is_defined_for_problem_type evalml.objectives.standard_metrics.LogLossMulticlass.objective_function evalml.objectives.standard_metrics.LogLossMulticlass.positive_only evalml.objectives.standard_metrics.LogLossMulticlass.score evalml.objectives.standard_metrics.LogLossMulticlass.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for log loss for multiclass classification. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: MAE Mean absolute error for regression. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(MAE().objective_function(y_true, y_pred), 0.2727272) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - MAE * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MAE.calculate_percent_difference evalml.objectives.standard_metrics.MAE.is_defined_for_problem_type evalml.objectives.standard_metrics.MAE.objective_function evalml.objectives.standard_metrics.MAE.positive_only evalml.objectives.standard_metrics.MAE.score evalml.objectives.standard_metrics.MAE.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for mean absolute error for regression. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: MAPE Mean absolute percentage error for time series regression. Scaled by 100 to return a percentage. Only valid for nonzero inputs. Otherwise, will throw a ValueError. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(MAPE().objective_function(y_true, y_pred), 15.9848484) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - Mean Absolute Percentage Error * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MAPE.calculate_percent_difference evalml.objectives.standard_metrics.MAPE.is_defined_for_problem_type evalml.objectives.standard_metrics.MAPE.objective_function evalml.objectives.standard_metrics.MAPE.positive_only evalml.objectives.standard_metrics.MAPE.score evalml.objectives.standard_metrics.MAPE.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for mean absolute percentage error for time series regression. .. py:method:: positive_only(self) If True, this objective is only valid for positive data. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: MaxError Maximum residual error for regression. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(MaxError().objective_function(y_true, y_pred), 1.0) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - MaxError * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MaxError.calculate_percent_difference evalml.objectives.standard_metrics.MaxError.is_defined_for_problem_type evalml.objectives.standard_metrics.MaxError.objective_function evalml.objectives.standard_metrics.MaxError.positive_only evalml.objectives.standard_metrics.MaxError.score evalml.objectives.standard_metrics.MaxError.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for maximum residual error for regression. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: MCCBinary Matthews correlation coefficient for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(MCCBinary().objective_function(y_true, y_pred), 0.2390457) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - True * - **is_bounded_like_percentage** - False * - **name** - MCC Binary * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MCCBinary.calculate_percent_difference evalml.objectives.standard_metrics.MCCBinary.can_optimize_threshold evalml.objectives.standard_metrics.MCCBinary.decision_function evalml.objectives.standard_metrics.MCCBinary.is_defined_for_problem_type evalml.objectives.standard_metrics.MCCBinary.objective_function evalml.objectives.standard_metrics.MCCBinary.optimize_threshold evalml.objectives.standard_metrics.MCCBinary.positive_only evalml.objectives.standard_metrics.MCCBinary.score evalml.objectives.standard_metrics.MCCBinary.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for Matthews correlation coefficient for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: MCCMulticlass Matthews correlation coefficient for multiclass classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(MCCMulticlass().objective_function(y_true, y_pred), 0.325) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - True * - **is_bounded_like_percentage** - False * - **name** - MCC Multiclass * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MCCMulticlass.calculate_percent_difference evalml.objectives.standard_metrics.MCCMulticlass.is_defined_for_problem_type evalml.objectives.standard_metrics.MCCMulticlass.objective_function evalml.objectives.standard_metrics.MCCMulticlass.positive_only evalml.objectives.standard_metrics.MCCMulticlass.score evalml.objectives.standard_metrics.MCCMulticlass.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for Matthews correlation coefficient for multiclass classification. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: MeanSquaredLogError Mean squared log error for regression. Only valid for nonnegative inputs. Otherwise, will throw a ValueError. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(MeanSquaredLogError().objective_function(y_true, y_pred), 0.0171353) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - Mean Squared Log Error * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MeanSquaredLogError.calculate_percent_difference evalml.objectives.standard_metrics.MeanSquaredLogError.is_defined_for_problem_type evalml.objectives.standard_metrics.MeanSquaredLogError.objective_function evalml.objectives.standard_metrics.MeanSquaredLogError.positive_only evalml.objectives.standard_metrics.MeanSquaredLogError.score evalml.objectives.standard_metrics.MeanSquaredLogError.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for mean squared log error for regression. .. py:method:: positive_only(self) If True, this objective is only valid for positive data. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: MedianAE Median absolute error for regression. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(MedianAE().objective_function(y_true, y_pred), 0.25) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - MedianAE * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MedianAE.calculate_percent_difference evalml.objectives.standard_metrics.MedianAE.is_defined_for_problem_type evalml.objectives.standard_metrics.MedianAE.objective_function evalml.objectives.standard_metrics.MedianAE.positive_only evalml.objectives.standard_metrics.MedianAE.score evalml.objectives.standard_metrics.MedianAE.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for median absolute error for regression. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: MSE Mean squared error for regression. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(MSE().objective_function(y_true, y_pred), 0.1590909) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - MSE * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.MSE.calculate_percent_difference evalml.objectives.standard_metrics.MSE.is_defined_for_problem_type evalml.objectives.standard_metrics.MSE.objective_function evalml.objectives.standard_metrics.MSE.positive_only evalml.objectives.standard_metrics.MSE.score evalml.objectives.standard_metrics.MSE.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for mean squared error for regression. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: Precision Precision score for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(Precision().objective_function(y_true, y_pred), 1.0) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Precision * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.Precision.calculate_percent_difference evalml.objectives.standard_metrics.Precision.can_optimize_threshold evalml.objectives.standard_metrics.Precision.decision_function evalml.objectives.standard_metrics.Precision.is_defined_for_problem_type evalml.objectives.standard_metrics.Precision.objective_function evalml.objectives.standard_metrics.Precision.optimize_threshold evalml.objectives.standard_metrics.Precision.positive_only evalml.objectives.standard_metrics.Precision.score evalml.objectives.standard_metrics.Precision.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for precision score for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: PrecisionMacro Precision score for multiclass classification using macro-averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(PrecisionMacro().objective_function(y_true, y_pred), 0.5555555) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Precision Macro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.PrecisionMacro.calculate_percent_difference evalml.objectives.standard_metrics.PrecisionMacro.is_defined_for_problem_type evalml.objectives.standard_metrics.PrecisionMacro.objective_function evalml.objectives.standard_metrics.PrecisionMacro.positive_only evalml.objectives.standard_metrics.PrecisionMacro.score evalml.objectives.standard_metrics.PrecisionMacro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for precision score for multiclass classification using macro-averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: PrecisionMicro Precision score for multiclass classification using micro averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(PrecisionMicro().objective_function(y_true, y_pred), 0.5454545) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Precision Micro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.PrecisionMicro.calculate_percent_difference evalml.objectives.standard_metrics.PrecisionMicro.is_defined_for_problem_type evalml.objectives.standard_metrics.PrecisionMicro.objective_function evalml.objectives.standard_metrics.PrecisionMicro.positive_only evalml.objectives.standard_metrics.PrecisionMicro.score evalml.objectives.standard_metrics.PrecisionMicro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for precision score for binary classification using micro-averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: PrecisionWeighted Precision score for multiclass classification using weighted averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(PrecisionWeighted().objective_function(y_true, y_pred), 0.5606060) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Precision Weighted * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.PrecisionWeighted.calculate_percent_difference evalml.objectives.standard_metrics.PrecisionWeighted.is_defined_for_problem_type evalml.objectives.standard_metrics.PrecisionWeighted.objective_function evalml.objectives.standard_metrics.PrecisionWeighted.positive_only evalml.objectives.standard_metrics.PrecisionWeighted.score evalml.objectives.standard_metrics.PrecisionWeighted.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for precision score for multiclass classification using weighted averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: R2 Coefficient of determination for regression. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(R2().objective_function(y_true, y_pred), 0.7638036) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - True * - **is_bounded_like_percentage** - False * - **name** - R2 * - **perfect_score** - 1 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.R2.calculate_percent_difference evalml.objectives.standard_metrics.R2.is_defined_for_problem_type evalml.objectives.standard_metrics.R2.objective_function evalml.objectives.standard_metrics.R2.positive_only evalml.objectives.standard_metrics.R2.score evalml.objectives.standard_metrics.R2.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for coefficient of determination for regression. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: Recall Recall score for binary classification. .. rubric:: Example >>> y_true = pd.Series([0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 1]) >>> y_pred = pd.Series([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1]) >>> np.testing.assert_almost_equal(Recall().objective_function(y_true, y_pred), 0.1428571) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Recall * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.Recall.calculate_percent_difference evalml.objectives.standard_metrics.Recall.can_optimize_threshold evalml.objectives.standard_metrics.Recall.decision_function evalml.objectives.standard_metrics.Recall.is_defined_for_problem_type evalml.objectives.standard_metrics.Recall.objective_function evalml.objectives.standard_metrics.Recall.optimize_threshold evalml.objectives.standard_metrics.Recall.positive_only evalml.objectives.standard_metrics.Recall.score evalml.objectives.standard_metrics.Recall.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for recall score for binary classification. .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring. .. py:class:: RecallMacro Recall score for multiclass classification using macro averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(RecallMacro().objective_function(y_true, y_pred), 0.5555555) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Recall Macro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.RecallMacro.calculate_percent_difference evalml.objectives.standard_metrics.RecallMacro.is_defined_for_problem_type evalml.objectives.standard_metrics.RecallMacro.objective_function evalml.objectives.standard_metrics.RecallMacro.positive_only evalml.objectives.standard_metrics.RecallMacro.score evalml.objectives.standard_metrics.RecallMacro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for recall score for multiclass classification using macro-averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: RecallMicro Recall score for multiclass classification using micro averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(RecallMicro().objective_function(y_true, y_pred), 0.5454545) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Recall Micro * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.RecallMicro.calculate_percent_difference evalml.objectives.standard_metrics.RecallMicro.is_defined_for_problem_type evalml.objectives.standard_metrics.RecallMicro.objective_function evalml.objectives.standard_metrics.RecallMicro.positive_only evalml.objectives.standard_metrics.RecallMicro.score evalml.objectives.standard_metrics.RecallMicro.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for recall score for multiclass classification using micro-averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: RecallWeighted Recall score for multiclass classification using weighted averaging. .. rubric:: Example >>> y_true = pd.Series([0, 1, 0, 2, 0, 1, 2, 1, 2, 0, 2]) >>> y_pred = pd.Series([0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2]) >>> np.testing.assert_almost_equal(RecallWeighted().objective_function(y_true, y_pred), 0.5454545) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - [0, 1] * - **greater_is_better** - True * - **is_bounded_like_percentage** - True * - **name** - Recall Weighted * - **perfect_score** - 1.0 * - **problem_types** - [ProblemTypes.MULTICLASS, ProblemTypes.TIME_SERIES_MULTICLASS] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.RecallWeighted.calculate_percent_difference evalml.objectives.standard_metrics.RecallWeighted.is_defined_for_problem_type evalml.objectives.standard_metrics.RecallWeighted.objective_function evalml.objectives.standard_metrics.RecallWeighted.positive_only evalml.objectives.standard_metrics.RecallWeighted.score evalml.objectives.standard_metrics.RecallWeighted.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for recall score for multiclass classification using weighted averaging. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: RootMeanSquaredError Root mean squared error for regression. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(RootMeanSquaredError().objective_function(y_true, y_pred), 0.3988620) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - Root Mean Squared Error * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.RootMeanSquaredError.calculate_percent_difference evalml.objectives.standard_metrics.RootMeanSquaredError.is_defined_for_problem_type evalml.objectives.standard_metrics.RootMeanSquaredError.objective_function evalml.objectives.standard_metrics.RootMeanSquaredError.positive_only evalml.objectives.standard_metrics.RootMeanSquaredError.score evalml.objectives.standard_metrics.RootMeanSquaredError.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for root mean squared error for regression. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed. .. py:class:: RootMeanSquaredLogError Root mean squared log error for regression. Only valid for nonnegative inputs. Otherwise, will throw a ValueError. .. rubric:: Example >>> y_true = pd.Series([1.5, 2, 3, 1, 0.5, 1, 2.5, 2.5, 1, 0.5, 2]) >>> y_pred = pd.Series([1.5, 2.5, 2, 1, 0.5, 1, 3, 2.25, 0.75, 0.25, 1.75]) >>> np.testing.assert_almost_equal(RootMeanSquaredLogError().objective_function(y_true, y_pred), 0.13090204) **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - False * - **is_bounded_like_percentage** - False * - **name** - Root Mean Squared Log Error * - **perfect_score** - 0.0 * - **problem_types** - [ProblemTypes.REGRESSION, ProblemTypes.TIME_SERIES_REGRESSION] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.standard_metrics.RootMeanSquaredLogError.calculate_percent_difference evalml.objectives.standard_metrics.RootMeanSquaredLogError.is_defined_for_problem_type evalml.objectives.standard_metrics.RootMeanSquaredLogError.objective_function evalml.objectives.standard_metrics.RootMeanSquaredLogError.positive_only evalml.objectives.standard_metrics.RootMeanSquaredLogError.score evalml.objectives.standard_metrics.RootMeanSquaredLogError.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, X=None, sample_weight=None) Objective function for root mean squared log error for regression. .. py:method:: positive_only(self) If True, this objective is only valid for positive data. .. py:method:: score(self, y_true, y_predicted, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validates the input based on a few simple checks. :param y_predicted: Predicted values of length [n_samples]. :type y_predicted: pd.Series, or pd.DataFrame :param y_true: Actual class labels of length [n_samples]. :type y_true: pd.Series :raises ValueError: If the inputs are malformed.