evalml.model_understanding.calculate_permutation_importance

evalml.model_understanding.calculate_permutation_importance(pipeline, X, y, objective, n_repeats=5, n_jobs=None, random_state=None, random_seed=0)[source]

Calculates permutation importance for features.

Parameters

  • pipeline (PipelineBase or subclass) – Fitted pipeline

  • X (ww.DataTable, pd.DataFrame) – The input data used to score and compute permutation importance

  • y (ww.DataColumn, pd.Series) – The target data

  • objective (str, ObjectiveBase) – Objective to score on

  • n_repeats (int) – Number of times to permute a feature. Defaults to 5.

  • n_jobs (int or None) – Non-negative integer describing level of parallelism used for pipelines. None and 1 are equivalent. If set to -1, all CPUs are used. For n_jobs below -1, (n_cpus + 1 + n_jobs) are used.

  • random_state (None, int) – Deprecated - use random_seed instead.

  • random_seed (int) – Seed for the random number generator. Defaults to 0.

Returns

pd.DataFrame, Mean feature importance scores over 5 shuffles.

evalml.model_understanding.roc_curve evalml.model_understanding.binary_objective_vs_threshold