cost_benefit_matrix =============================================== .. py:module:: evalml.objectives.cost_benefit_matrix .. autoapi-nested-parse:: Cost-benefit matrix objective. Module Contents --------------- Classes Summary ~~~~~~~~~~~~~~~ .. autoapisummary:: evalml.objectives.cost_benefit_matrix.CostBenefitMatrix Contents ~~~~~~~~~~~~~~~~~~~ .. py:class:: CostBenefitMatrix(true_positive, true_negative, false_positive, false_negative) Score using a cost-benefit matrix. Scores quantify the benefits of a given value, so greater numeric scores represents a better score. Costs and scores can be negative, indicating that a value is not beneficial. For example, in the case of monetary profit, a negative cost and/or score represents loss of cash flow. :param true_positive: Cost associated with true positive predictions. :type true_positive: float :param true_negative: Cost associated with true negative predictions. :type true_negative: float :param false_positive: Cost associated with false positive predictions. :type false_positive: float :param false_negative: Cost associated with false negative predictions. :type false_negative: float **Attributes** .. list-table:: :widths: 15 85 :header-rows: 0 * - **expected_range** - None * - **greater_is_better** - True * - **is_bounded_like_percentage** - False * - **name** - Cost Benefit Matrix * - **perfect_score** - None * - **problem_types** - [ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY] * - **score_needs_proba** - False **Methods** .. autoapisummary:: :nosignatures: evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.calculate_percent_difference evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.can_optimize_threshold evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.decision_function evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.is_defined_for_problem_type evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.objective_function evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.optimize_threshold evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.positive_only evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.score evalml.objectives.cost_benefit_matrix.CostBenefitMatrix.validate_inputs .. py:method:: calculate_percent_difference(cls, score, baseline_score) :classmethod: Calculate the percent difference between scores. :param score: A score. Output of the score method of this objective. :type score: float :param baseline_score: A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator. :type baseline_score: float :returns: The percent difference between the scores. Note that for objectives that can be interpreted as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score. :rtype: float .. py:method:: can_optimize_threshold(cls) :property: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true. :returns: Whether or not an objective can be optimized. :rtype: bool .. py:method:: decision_function(self, ypred_proba, threshold=0.5, X=None) Apply a learned threshold to predicted probabilities to get predicted classes. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series, np.ndarray :param threshold: Threshold used to make a prediction. Defaults to 0.5. :type threshold: float, optional :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: predictions .. py:method:: is_defined_for_problem_type(cls, problem_type) :classmethod: Returns whether or not an objective is defined for a problem type. .. py:method:: objective_function(self, y_true, y_predicted, y_train=None, X=None, sample_weight=None) Calculates cost-benefit of the using the predicted and true values. :param y_predicted: Predicted labels. :type y_predicted: pd.Series :param y_true: True labels. :type y_true: pd.Series :param y_train: Ignored. :type y_train: pd.Series :param X: Ignored. :type X: pd.DataFrame :param sample_weight: Ignored. :type sample_weight: pd.DataFrame :returns: Cost-benefit matrix score :rtype: float .. py:method:: optimize_threshold(self, ypred_proba, y_true, X=None) Learn a binary classification threshold which optimizes the current objective. :param ypred_proba: The classifier's predicted probabilities :type ypred_proba: pd.Series :param y_true: The ground truth for the predictions. :type y_true: pd.Series :param X: Any extra columns that are needed from training data. :type X: pd.DataFrame, optional :returns: Optimal threshold for this objective. :raises RuntimeError: If objective cannot be optimized. .. py:method:: positive_only(cls) If True, this objective is only valid for positive data. Defaults to False. .. py:method:: score(self, y_true, y_predicted, y_train=None, X=None, sample_weight=None) Returns a numerical score indicating performance based on the differences between the predicted and actual values. :param y_predicted: Predicted values of length [n_samples] :type y_predicted: pd.Series :param y_true: Actual class labels of length [n_samples] :type y_true: pd.Series :param y_train: Observed training values of length [n_samples] :type y_train: pd.Series :param X: Extra data of shape [n_samples, n_features] necessary to calculate score :type X: pd.DataFrame or np.ndarray :param sample_weight: Sample weights used in computing objective value result :type sample_weight: pd.DataFrame or np.ndarray :returns: score .. py:method:: validate_inputs(self, y_true, y_predicted) Validate inputs for scoring.