{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set the loglevel to critical to reduce the amount of logs you will see while running this tutorial\n",
    "import logging\n",
    "logger = logging.getLogger('leip_recipe_designer.tasks')\n",
    "logger.setLevel(logging.CRITICAL)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# From Classifier GRDB to Deployable Artifacts: A Step-by-Step Guide\n",
    "\n",
    "This How-To Guide walks you through the process of loading a recipe from the Classifier GRDB, training the model, and generating deployable artifacts. We'll use the tools available in LEIP Design to perform this workflow step-by-step.\n",
    "\n",
    "---\n",
    "\n",
    "## **Overview**\n",
    "\n",
    "In this guide, we will:\n",
    "1. Load a recipe from the Classifier GRDB.\n",
    "2. Train the selected model with a dataset.\n",
    "3. Evaluate the trained model's performance.\n",
    "4. Optimize and compile the model for a hardware target.\n",
    "5. Export the deployable model and stub code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To begin, let's load a recipe with ID 4163 from the classifier GRDB. The API for this differs slightly from what you've encountered with detectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import leip_recipe_designer as rd\n",
    "\n",
    "workspace = Path('./workspace')\n",
    "pantry = rd.Pantry.build(workspace / \"./my_combined_pantry/\", force_rebuild=False)\n",
    "recipe = rd.create.from_recipe_id(\"4163\", pantry=pantry, task=\"vision.classification.2d\", volume=\"xval_cls\", default=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this guide, we will use the EuroSAT dataset, a public dataset of satellite images. The dataset can be downloaded and prepared for use with the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = rd.helpers.data.get_data_generator_by_name(pantry=pantry, regex_ingredient_name=\"EuroSAT\", category=\"classification\")\n",
    "rd.helpers.data.replace_data_generator(recipe, data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the dataset is ready, we need to configure the recipe to use it. Then, we will train the model for a single epoch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "recipe[\"train.num_epochs\"] = 1\n",
    "recipe.fill_empty_recursively()\n",
    "train_output = rd.tasks.train(recipe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After training, it is essential to evaluate the model to measure its performance. This will provide insights into its accuracy, and other metrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Assign a checkpoint ingredient to the recipe for later use (e.g., evaluation)\n",
    "recipe.assign_ingredients(\"checkpoint\", \"Local ckpt file\")\n",
    "recipe['checkpoint.path'] = str(train_output[\"best_model_path\"])\n",
    "\n",
    "# Evaluate the trained recipe\n",
    "eval_output = rd.tasks.evaluate(recipe)\n",
    "\n",
    "from pprint import pprint\n",
    "\n",
    "print(\"Evaluation Metrics:\")\n",
    "pprint(eval_output['evaluate.metric_dict'], indent=2, width=80, compact=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To deploy the model on specific hardware, it must be optimized and compiled. For example, you can target GPU with CUDA:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "compiler: TensorRT Compiler (slot:compiler, id:f71957, version:1.0.1)\n",
      "  compiler.set_channel_layout  =  None               (slot:set_channel_layout)\n",
      "     compiler.export_metadata  =  False              (slot:export_metadata)\n",
      "     compiler.force_overwrite  =  False              (slot:force_overwrite)\n",
      "        compiler.export_relay  =  False              (slot:export_relay)\n",
      "         compiler.output_path  =  ./compile_output   (slot:output_path)\n",
      "         compiler.set_float16  =  False              (slot:set_float16)\n",
      "           compiler.opt_level  =  3                  (slot:opt_level)\n",
      "              compiler.target  =  nvidia/rtx-a4500   (slot:target)\n",
      "                compiler.host  =  intel/cascadelake  (slot:host)\n",
      "  quantizer: No Quantizer (slot:quantizer, id:c00837, version:1.0.0)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(recipe['compiler'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "recipe['compiler.output_path'] = './rd_forge_output'\n",
    "recipe['compiler.force_overwrite'] = True\n",
    "\n",
    "compiled_model = rd.tasks.compile(recipe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Evaluate the compiled model (just on a single batch) as you have seen already in our getting started tutorial!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "recipe.assign_ingredients(\"checkpoint\", \"compiled\")\n",
    "recipe['checkpoint.path'] = str(compiled_model['compiler.output_file'])\n",
    "\n",
    "# Set the validation batch size to match the compiled batch size\n",
    "recipe[\"train.batch_size_val\"] = recipe[\"export.batch_size\"]\n",
    "\n",
    "# Limit the number of evaluation batches to speed up the example evaluation\n",
    "recipe['evaluation.max_batches'] = 1\n",
    "\n",
    "# Evaluate the compiled recipe\n",
    "compiled_eval_output = rd.tasks.evaluate(recipe)\n",
    "\n",
    "# Print accuracy after compilation\n",
    "print(f\"Accuracy after compilation: {compiled_eval_output['evaluate.metric_single']:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we will generate the necessary stub code for inference, along with the compiled model artifact. This package will include everything required to deploy the model on your target edge hardware."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "recipe['model.use_pretrained'] = False\n",
    "demo_dir = rd.tasks.generate_stub_code(recipe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, you can use the generated files to deploy your model on the desired hardware platform, such as a CUDA-enabled GPU, based on your specified target configuration. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Welcome to LEIP Design, that's how we make advanced ML workflows accessible to everyone!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "design",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}