{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c67bb1df",
   "metadata": {},
   "source": [
    "# Using PyLRE to Deploy your Optimized model\n",
    "\n",
    "You have optimized your trained model using LEIP Optimize, and now you want to deploy it in a target environment. This tutorial provides a quick start guide for loading an optimized artifact, creating an LRE instance, and performing inference."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d5a0bf3",
   "metadata": {},
   "source": [
    "## Runtime Setup\n",
    "\n",
    "We need two components to execute a model on your target:\n",
    "- a target-compatible and model-compatible runtime (LRE)\n",
    "- a target-compatible model or model library (optimized output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bb8c64de",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pylre import LatentRuntimeEngine as LRE\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "627a63af",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimized_artifact_path = \"path/to/optimized_model.onnx\" #  or \"path/to/optimized_model/modelLibrary.so\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87debe03",
   "metadata": {},
   "outputs": [],
   "source": [
    "lre = LRE(optimized_artifact_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2019ace1",
   "metadata": {},
   "source": [
    "## Create a random tensor to test inference\n",
    "To verify that the LRE instantiation is working correctly, we can feed it a randomly generated input tensor. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "093c9326",
   "metadata": {},
   "outputs": [],
   "source": [
    "def _normalize_shape(s):\n",
    "    \"\"\"\n",
    "    Normalize shape tuple by replacing dynamic dimensions (None or negative) with 1.\n",
    "    \n",
    "    Args:\n",
    "        s: Shape tuple that may contain None or negative values for dynamic dimensions\n",
    "        \n",
    "    Returns:\n",
    "        Tuple with all dynamic dimensions replaced with 1\n",
    "    \"\"\"\n",
    "    return tuple(1 if dim is None or dim < 0 else dim for dim in s)\n",
    "\n",
    "shapes = lre.input_shapes\n",
    "dtypes = lre.input_dtypes\n",
    "\n",
    "input = [\n",
    "    np.random.random(_normalize_shape(shape)).astype(np.dtype(dtype))\n",
    "    for shape, dtype in zip(shapes, dtypes)\n",
    "]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32f07729",
   "metadata": {},
   "source": [
    "With this input data tensor, we can run an inference on the model LRE instantiation we created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed84c41c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ensure outputs are on CPU since we'll be using NumPy for post-processing with from_dlpack\n",
    "lre.set_cpu_output(True)\n",
    "output = lre(input)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "876cdb2a",
   "metadata": {},
   "source": [
    "This output is in a device-independent format. You may want to convert it to a format that's easier to work with for postprocessing. We'll use NumPy in this example, but depending on your application and hardware, other formats may be more suitable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1025a59",
   "metadata": {},
   "outputs": [],
   "source": [
    "numpy_output = [np.from_dlpack(o) for o in output]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pylre-dev",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}