sensitivity_low_alert¶

Module Contents¶

Classes Summary¶

SensitivityLowAlert

Base class for all binary classification objectives.

Attributes Summary¶

logger

Contents¶

evalml.objectives.sensitivity_low_alert.logger¶

class evalml.objectives.sensitivity_low_alert.SensitivityLowAlert(alert_rate=0.01)[source]¶

Base class for all binary classification objectives.

Attributes

greater_is_better	True
is_bounded_like_percentage	True
name	Sensitivity at Low Alert Rates
perfect_score	1.0
problem_types	[ProblemTypes.BINARY, ProblemTypes.TIME_SERIES_BINARY]
score_needs_proba	False

Methods

`calculate_percent_difference`	Calculate the percent difference between scores.
`can_optimize_threshold`	Returns a boolean determining if we can optimize the binary classification objective threshold.
`decision_function`	Determine if an observation is high risk given an alert rate
`is_defined_for_problem_type`
`objective_function`	Calculate sensitivity across all predictions, using the top alert_rate percent of observations as the predicted positive class
`optimize_threshold`	Learn a binary classification threshold which optimizes the current objective.
`positive_only`	If True, this objective is only valid for positive data. Default False.
`score`	Returns a numerical score indicating performance based on the differences between the predicted and actual values.
`validate_inputs`	Validates the input based on a few simple checks.

classmethod calculate_percent_difference(cls, score, baseline_score)¶

Calculate the percent difference between scores.

Parameters

score (float) – A score. Output of the score method of this objective.
baseline_score (float) – A score. Output of the score method of this objective. In practice, this is the score achieved on this objective with a baseline estimator.

Returns

The percent difference between the scores. Note that for objectives that can be interpreted: as percentages, this will be the difference between the reference score and score. For all other objectives, the difference will be normalized by the reference score.

Return type

float

property can_optimize_threshold(cls)¶: Returns a boolean determining if we can optimize the binary classification objective threshold. This will be false for any objective that works directly with predicted probabilities, like log loss and AUC. Otherwise, it will be true.

decision_function(self, ypred_proba, **kwargs)[source]¶

Determine if an observation is high risk given an alert rate

Parameters: ypred_proba (pd.Series) – Predicted probabilities

classmethod is_defined_for_problem_type(cls, problem_type)¶

objective_function(self, y_true, y_predicted, **kwargs)[source]¶

Calculate sensitivity across all predictions, using the top alert_rate percent of observations as the predicted positive class

Parameters

y_true (pd.Series) – True labels
y_predicted (pd.Series) – Predicted labels based on alert_rate

Returns

sensitivity using the observations with the top scores as the predicted positive class

Return type

float

optimize_threshold(self, ypred_proba, y_true, X=None)¶

Learn a binary classification threshold which optimizes the current objective.

Parameters

ypred_proba (pd.Series) – The classifier’s predicted probabilities
y_true (pd.Series) – The ground truth for the predictions.
X (pd.DataFrame, optional) – Any extra columns that are needed from training data.

Returns

Optimal threshold for this objective

positive_only(cls)¶: If True, this objective is only valid for positive data. Default False.

score(self, y_true, y_predicted, X=None, sample_weight=None)¶

Returns a numerical score indicating performance based on the differences between the predicted and actual values.

Parameters

y_predicted (pd.Series) – Predicted values of length [n_samples]
y_true (pd.Series) – Actual class labels of length [n_samples]
X (pd.DataFrame or np.ndarray) – Extra data of shape [n_samples, n_features] necessary to calculate score
sample_weight (pd.DataFrame or np.ndarray) – Sample weights used in computing objective value result

Returns

score

validate_inputs(self, y_true, y_predicted)¶

Validates the input based on a few simple checks.

Parameters

y_predicted (pd.Series, or pd.DataFrame) – Predicted values of length [n_samples]
y_true (pd.Series) – Actual class labels of length [n_samples]

Returns

None

regression_objective standard_metrics