{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Start"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this guide, we'll show how you can use EvalML to automatically find the best pipeline for predicting whether or not a credit card transaction is fradulent. Along the way, we'll highlight EvalML's built-in tools and features for understanding and interacting with the search process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import evalml\n",
    "from evalml import AutoMLSearch\n",
    "from evalml.utils import infer_feature_types"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we load in the features and outcomes we want to use to train our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X, y = evalml.demos.load_fraud(n_rows=250)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we will clean the data. Since EvalML accepts a pandas input, it can run type inference on this data directly. Since we'd like to change the types inferred by EvalML, we can use the `infer_feature_types` utility method. Here's what we're going to do with the following dataset:\n",
    "\n",
    "- Reformat the `expiration_date` column so it reflects a more familiar date format.\n",
    "- Cast the `lat` and `lng` columns from float to str.\n",
    "- Use `infer_feature_types` to specify what types certain columns should be. For example, to avoid having the `provider` column be inferred as natural language text, we have specified it as a categorical column instead.\n",
    "\n",
    "The `infer_feature_types` utility method takes a pandas or numpy input and converts it to a pandas dataframe with a [Woodwork](https://woodwork.alteryx.com/en/stable/) accessor, providing us with flexibility to cast the data as necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X.ww[\"expiration_date\"] = X[\"expiration_date\"].apply(\n",
    "    lambda x: \"20{}-01-{}\".format(x.split(\"/\")[1], x.split(\"/\")[0])\n",
    ")\n",
    "X = infer_feature_types(\n",
    "    X,\n",
    "    feature_types={\n",
    "        \"store_id\": \"categorical\",\n",
    "        \"expiration_date\": \"datetime\",\n",
    "        \"lat\": \"categorical\",\n",
    "        \"lng\": \"categorical\",\n",
    "        \"provider\": \"categorical\",\n",
    "    },\n",
    ")\n",
    "X.ww"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to validate the results of the pipeline creation and optimization process, we will save some of our data as a holdout set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_holdout, y_train, y_holdout = evalml.preprocessing.split_data(\n",
    "    X, y, problem_type=\"binary\", test_size=0.2\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Note:__ To provide data to EvalML, it is recommended that you initialize a woodwork accessor so that you control how EvalML will treat each feature, such as as a numeric feature, a categorical feature, a text feature or other type of feature. Consult the [the Woodwork project](https://woodwork.alteryx.com/en/stable/) for help on how to do this. Here, `split_data()` returns dataframes with woodwork accessors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "EvalML has many options to configure the pipeline search. At the minimum, we need to define an objective function. For simplicity, we will use the F1 score in this example. However, the real power of EvalML is in using domain-specific [objective functions](user_guide/objectives.ipynb) or [building your own](https://evalml.alteryx.com/en/stable/user_guide/objectives.html#Custom-Objectives).\n",
    "\n",
    "Below EvalML utilizes Bayesian optimization (EvalML's default optimizer) to search and find the best pipeline defined by the given objective.\n",
    "\n",
    "EvalML provides a number of parameters to control the search process. `max_batches` is one of the parameters which controls the stopping criterion for the AutoML search. It indicates the maximum number of rounds of AutoML to evaluate, where each round may train and score a variable number of pipelines. In this example, `max_batches` is set to 1.\n",
    "\n",
    "** Graphing methods, like AutoMLSearch, on Jupyter Notebook and Jupyter Lab require [ipywidgets](https://ipywidgets.readthedocs.io/en/latest/user_install.html) to be installed.\n",
    "\n",
    "** If graphing on Jupyter Lab, [jupyterlab-plotly](https://plotly.com/python/getting-started/#jupyterlab-support-python-35) required. To download this, make sure you have [npm](https://nodejs.org/en/download/) installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "automl = AutoMLSearch(\n",
    "    X_train=X_train,\n",
    "    y_train=y_train,\n",
    "    problem_type=\"binary\",\n",
    "    objective=\"f1\",\n",
    "    max_batches=2,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we call `search()`, the search for the best pipeline will begin. There is no need to wrangle with missing data or categorical variables as EvalML includes various preprocessing steps (like imputation, one-hot encoding, feature selection) to ensure you're getting the best results. As long as your data is in a single table, EvalML can handle it. If not, you can reduce your data to a single table by utilizing [Featuretools](https://featuretools.featurelabs.com) and its Entity Sets.\n",
    "\n",
    "You can find more information on pipeline components and how to integrate your own custom pipelines into EvalML [here](user_guide/pipelines.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "automl.search(interactive_plot=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you would like to suppress stdout output, set `verbose=False`. This is also the default behavior for `AutoMLSearch` if `verbose` is not specified. \n",
    "\n",
    "Also, if you would like to see the interactive plot update dynamically over time as the search progresses, either remove the parameter or set `interactive_plot=True`. This is the default setting for `search()` if `interactive_plot` is not specified (it is set to False here due to documentation workaround)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "automl = AutoMLSearch(\n",
    "    X_train=X_train,\n",
    "    y_train=y_train,\n",
    "    problem_type=\"binary\",\n",
    "    objective=\"f1\",\n",
    "    max_batches=2,\n",
    "    verbose=False,\n",
    ")\n",
    "automl.search()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also provide a standalone `search` [method](./autoapi/evalml/automl/index.rst#evalml.automl.search) which does all of the above in a single line, and returns the `AutoMLSearch` instance and data check results. If there were data check errors, AutoML will not be run and no `AutoMLSearch` instance will be returned."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the search is finished we can view all of the pipelines searched, ranked by score. Internally, EvalML performs cross validation to score the pipelines. If it notices a high variance across cross validation folds, it will warn you. EvalML also provides additional [data checks](user_guide/data_checks.ipynb) to analyze your data to assist you in producing the best performing pipeline. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "automl.rankings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we are interested in see more details about the pipeline, we can view a summary description using the `id` from the rankings table:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "automl.describe_pipeline(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also view the pipeline parameters directly:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline = automl.get_pipeline(3)\n",
    "print(pipeline.parameters)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now select the best pipeline and score it on our holdout data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline = automl.best_pipeline\n",
    "pipeline.score(X_holdout, y_holdout, [\"f1\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also visualize the structure of the components contained by the pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline.graph()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}