engine_base¶
Base class for EvalML engines.
Module Contents¶
Classes Summary¶
Base class for EvalML engines. |
|
Wrapper around the result of a (possibly asynchronous) engine computation. |
|
Mimic the behavior of a python logging.Logger but stores all messages rather than actually logging them. |
Functions¶
Function submitted to the submit_evaluation_job engine method. |
|
Wrap around pipeline.score method to make it easy to score pipelines with dask. |
|
Given a pipeline, config and data, train and score the pipeline and return the CV or TV scores. |
|
Train a pipeline and tune the threshold if necessary. |
Contents¶
-
class
evalml.automl.engine.engine_base.
EngineBase
[source]¶ Base class for EvalML engines.
Methods
Set up logger for job.
Submit job for pipeline evaluation during AutoMLSearch.
Submit job for pipeline scoring.
Submit job for pipeline training.
-
abstract
submit_evaluation_job
(self, automl_config, pipeline, X, y)[source]¶ Submit job for pipeline evaluation during AutoMLSearch.
-
abstract
-
class
evalml.automl.engine.engine_base.
EngineComputation
[source]¶ Wrapper around the result of a (possibly asynchronous) engine computation.
Methods
Cancel the computation.
Whether the computation is done.
Gets the computation result. Will block until the computation is finished.
-
evalml.automl.engine.engine_base.
evaluate_pipeline
(pipeline, automl_config, X, y, logger)[source]¶ Function submitted to the submit_evaluation_job engine method.
- Parameters
pipeline (PipelineBase) – The pipeline to score.
automl_config (AutoMLConfig) – The AutoMLSearch object, used to access config and the error callback.
X (pd.DataFrame) – Training features.
y (pd.Series) – Training target.
logger – Logger object to write to.
- Returns
- First - A dict containing cv_score_mean, cv_scores, training_time and a cv_data structure with details.
Second - The pipeline class we trained and scored. Third - the job logger instance with all the recorded messages.
- Return type
tuple of three items
-
class
evalml.automl.engine.engine_base.
JobLogger
[source]¶ Mimic the behavior of a python logging.Logger but stores all messages rather than actually logging them.
This is used during engine jobs so that log messages are recorded after the job completes. This is desired so that all of the messages for a single job are grouped together in the log.
Methods
Store message at the debug level.
Store message at the error level.
Store message at the info level.
Store message at the warning level.
Write all the messages to the logger, first in, first out (FIFO) order.
-
evalml.automl.engine.engine_base.
score_pipeline
(pipeline, X, y, objectives, X_train=None, y_train=None, X_schema=None, y_schema=None)[source]¶ Wrap around pipeline.score method to make it easy to score pipelines with dask.
- Parameters
pipeline (PipelineBase) – The pipeline to score.
X (pd.DataFrame) – Features to score on.
y (pd.Series) – Target used to calculate scores.
objectives (list[ObjectiveBase]) – List of objectives to score on.
X_train (pd.DataFrame) – Training features. Used for feature engineering in time series.
y_train (pd.Series) – Training target. Used for feature engineering in time series.
X_schema (ww.TableSchema) – Schema for features. Defaults to None.
y_schema (ww.ColumnSchema) – Schema for columns. Defaults to None.
- Returns
Dictionary object containing pipeline scores.
- Return type
dict
-
evalml.automl.engine.engine_base.
train_and_score_pipeline
(pipeline, automl_config, full_X_train, full_y_train, logger)[source]¶ Given a pipeline, config and data, train and score the pipeline and return the CV or TV scores.
- Parameters
pipeline (PipelineBase) – The pipeline to score.
automl_config (AutoMLSearch) – The AutoMLSearch object, used to access config and the error callback.
full_X_train (pd.DataFrame) – Training features.
full_y_train (pd.Series) – Training target.
logger – Logger object to write to.
- Raises
Exception – If there are missing target values in the training set after data split.
- Returns
- First - A dict containing cv_score_mean, cv_scores, training_time and a cv_data structure with details.
Second - The pipeline class we trained and scored. Third - the job logger instance with all the recorded messages.
- Return type
tuple of three items
-
evalml.automl.engine.engine_base.
train_pipeline
(pipeline, X, y, automl_config, schema=True, get_hashes=False)[source]¶ Train a pipeline and tune the threshold if necessary.
- Parameters
pipeline (PipelineBase) – Pipeline to train.
X (pd.DataFrame) – Features to train on.
y (pd.Series) – Target to train on.
automl_config (AutoMLSearch) – The AutoMLSearch object, used to access config and the error callback.
schema (bool) – Whether to use the schemas for X and y. Defaults to True.
get_hashes (bool) – Whether to return the hashes of the data used to train (and potentially threshold). Defaults to False
- Returns
A trained pipeline instance. hash (optional): The hash of the input data indices, only returned when get_hashes is True.
- Return type
pipeline (PipelineBase)