Add in an example notebook for migrating models from SAS Viya 3.5 to SAS Viya 4.

Author: smlindauer

examples/data/ModelMigration/DecisionTreeClassifier.zip

@@ -0,0 +1,277 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "8ac109d6-4746-4ac6-befc-8fcd249f8739",
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "source": [
+    "Copyright © 2023, SAS Institute Inc., Cary, NC, USA.  All Rights Reserved.\n",
+    "SPDX-License-Identifier: Apache-2.0"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "417ee379-53a8-490a-9cb1-d06e6387dd3c",
+   "metadata": {
+    "Collapsed": "false"
+   },
+   "source": [
+    "# Model Migration: Moving Models from SAS Viya 3.5 to SAS Viya 4 in SAS Model Manager\n",
+    "\n",
+    "This notebook provides an example scenario for downloading a model from SAS Viya 3.5, converting it in to a format acceptable for SAS Viya 4, then uploads the model to a SAS Viya 4 server. In order to complete the conversion, the following files are modified:\n",
+    "- dmcas_epscorecode.sas (deleted)\n",
+    "- dmcas_packagescore.sas (deleted)\n",
+    "- score.sas (deleted)\n",
+    "- ModelProperties.json\n",
+    "- fileMetadata.json\n",
+    "- python score code file\n",
+    "\n",
+    "Over the course of this notebook, a model from SAS Viya 3.5 is downloaded to a specified directory. The model contents are modified to the format expected in SAS Viya 4, then rezipped into a single model zip file. This new model is then uploaded to a SAS Viya 4 server. Optionally, the model zip file can be deleted, leaving the directory in an empty state for the next model to be migrated."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "29860b48-27af-428a-9012-676711a6220a",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import io\n",
+    "import zipfile\n",
+    "from pathlib import Path\n",
+    "\n",
+    "from sasctl._services.model_repository import ModelRepository as mr\n",
+    "from sasctl.core import is_uuid\n",
+    "from sasctl.utils import convert_model_zip\n",
+    "from sasctl import Session"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d20270bb-6b3a-4e07-ae7c-6e3181ec2f11",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def set_model_directory(path=Path.cwd()):\n",
+    "    \"\"\"\n",
+    "    Set or create an empty directory path for model migration\n",
+    "\n",
+    "    Parameters\n",
+    "    ----------\n",
+    "    path : str or Path, optional\n",
+    "        A string or Path object pointing at an empty or nonexistent directory; default is the current working directory.\n",
+    "\n",
+    "    Returns\n",
+    "    -------\n",
+    "    dir_path : Path\n",
+    "        A Path object indicating the empty directory for models.\n",
+    "    Raises\n",
+    "    ------\n",
+    "    FileExistsError:\n",
+    "        The provided directory is not empty.\n",
+    "    \"\"\"\n",
+    "    # Check if provided path is a valid directory and is empty\n",
+    "    if Path(path).is_dir():\n",
+    "        # Check if directory is not empty\n",
+    "        if any(Path(path).iterdir()):\n",
+    "            raise FileExistsError(f\"The directory {str(Path(path))} is not empty. Please either provide\" +\n",
+    "                                  \" an empty directory path or path of a directory to be created.\")\n",
+    "        return path\n",
+    "    elif Path(path).is_file():\n",
+    "        print(\"The provided path points at a file. Attempting to create a new directory with the name of the file.\")\n",
+    "        new_path = set_model_directory(Path(path).parent / Path(path).stem)\n",
+    "        return new_path\n",
+    "    else:\n",
+    "        Path(path).mkdir(parents=True)\n",
+    "        print(f\"The {str(Path(path))} directory was created.\")\n",
+    "        return path"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a5dc813b-3ddc-4e45-946a-2920b16b3e58",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def download_viya35_model(path, model35, project35=None):\n",
+    "    \"\"\" Download the model zip for a SAS Viya 3.5 model from SAS Model Manager. Then unzip the model and delete the\n",
+    "    zip archive downloaded.\n",
+    "\n",
+    "    Parameters\n",
+    "    ----------\n",
+    "    path : Path object\n",
+    "        Parent directory for models to be migrated.\n",
+    "    model : str or dict\n",
+    "        The name or id of the model, or a dictionary representation of the model.\n",
+    "    project : str or dict, optional\n",
+    "        The name or id of the project, or a dictionary representation of the project. Default value is None.\n",
+    "\n",
+    "    Returns\n",
+    "    -------\n",
+    "    model : dict\n",
+    "        A dictionary representation of the model.\n",
+    "    project : dict or None\n",
+    "        A dictionary representation of the project.\n",
+    "    \"\"\"\n",
+    "    if project35:\n",
+    "        project35 = mr.get_project(project35)\n",
+    "        if is_uuid(model35):\n",
+    "            model35 = mr.get_model(model35)\n",
+    "        elif isinstance(model35, dict) and \"id\" in model35:\n",
+    "            model35 = model35\n",
+    "        else:\n",
+    "            model35 = mr.list_models(filter=f\"and(eq(projectName,'{project35.name}'),eq(name,'{model35}'))\")[0]\n",
+    "    else:\n",
+    "        # If only a name and no project is provided, the correct model may not be found from the repository\n",
+    "        if (not is_uuid(model35)) and (not isinstance(model35, dict)):\n",
+    "            print(\"No project was provided for model {}.\".format(model) + \" If another model with the same name\" +\n",
+    "                  \" exists in the repository, it is not guaranteed to be the model you are attempting to reference.\")\n",
+    "        model35 = mr.get_model(model35)\n",
+    "\n",
+    "    model_zip = mr.get(f\"models/{model35.id}\", params={\"format\": \"zip\"}, format_=\"content\")\n",
+    "    zip_path = Path(path) / (model35.name + \".zip\")\n",
+    "    with open(zip_path, \"wb\") as z_file:\n",
+    "        z_file.write(model_zip)\n",
+    "    print(f\"Model {model35.name} downloaded to {str(zip_path)}.\")\n",
+    "    with zipfile.ZipFile(str(zip_path), \"r\") as zip_contents:\n",
+    "        zip_contents.extractall(path)\n",
+    "    print(\"Model contents have been extracted from {}.\".format(str(zip_path)))\n",
+    "    zip_path.unlink()\n",
+    "    try:\n",
+    "        return model35, mr.get_project(model35.projectId)\n",
+    "    except AttributeError:\n",
+    "        return model35, None"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ce90b596-649c-48f0-b21a-6e20142c21f0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def convert_and_upload_model(path, model4, project4=None):\n",
+    "    \"\"\" Convert the model Viya 3.5 model contents to the Viya 4 model content format, then upload the model to a\n",
+    "    specified Viya 4 server.\n",
+    "\n",
+    "    Parameters\n",
+    "    ----------\n",
+    "    path : str or Path object\n",
+    "        A string or Path object where the Viya 3.5 model contents are located.\n",
+    "    model : str or dict\n",
+    "        The name or id of the model, or a dictionary representation of the model.\n",
+    "    project : str or dict, optional\n",
+    "        The name or id of the project, or a dictionary representation of the project. Default value is None.\n",
+    "    \"\"\"\n",
+    "    convert_model_zip(path)\n",
+    "    \n",
+    "    project4 = mr.get_project(project4)\n",
+    "\n",
+    "    # List all files in directory\n",
+    "    files = [x for x in path.glob(\"**/*\") if x.is_file()]\n",
+    "    # Zip all files in directory into a zip archive\n",
+    "    with zipfile.ZipFile(str(path / (model4 + \".zip\")), \"w\") as zip_file:\n",
+    "        for file in files:\n",
+    "            zip_file.write(str(file), arcname=file.name)\n",
+    "            file.unlink()\n",
+    "\n",
+    "    # Import the model into SAS Model Manager\n",
+    "    with open(path / (model4 + \".zip\"), \"rb\") as zip_file:\n",
+    "        if project4:\n",
+    "            response = mr.import_model_from_zip(model4, project4, io.BytesIO(zip_file.read()))\n",
+    "        else:\n",
+    "            params = {\"name\": model4.name, \"type\": \"ZIP\", \"versionOption\": \"latest\"}\n",
+    "            params = \"&\".join(\"{}={}\".format(k, v) for k, v in params.items())\n",
+    "            response = mr.post(\n",
+    "                \"/models#octetStream\",\n",
+    "                data=io.BytesIO(zip_file.read()).read(),\n",
+    "                params=params,\n",
+    "                headers={\"Content-Type\": \"application/octet-stream\"},\n",
+    "            )\n",
+    "    # Uncomment if you want to delete the converted model zip\n",
+    "    # Path(path / model4.name + \".zip\").unlink()\n",
+    "\n",
+    "    try:\n",
+    "        print(\n",
+    "            \"Model was successfully imported into SAS Model Manager as {} with UUID: {}.\".format(\n",
+    "                response.name, response.id\n",
+    "            )\n",
+    "        )\n",
+    "    except AttributeError:\n",
+    "        print(\"Model, {}, failed to import into SAS Model Manager.\".format(model.name))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "36ba4f92-c446-4481-9213-37a178d33d47",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model_name = \"ExampleModel\"\n",
+    "project_name = \"ExampleProject\"\n",
+    "viya35_host = \"demo3.5.sas.com\"\n",
+    "viya4_host = \"demo4.sas.com\"\n",
+    "username = \"sasuser\"\n",
+    "password = \"saspass\"\n",
+    "path = Path.cwd() / \"data/ModelMigration\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "9b0c93d8-93b7-4b98-a5e9-e7a21890e67d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "set_model_directory(path)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ba6a807a-85e2-4be8-b31c-77bf19bf5abb",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "with Session(viya35_host, username, password, protocol=\"http\"):\n",
+    "    model, project = download_viya35_model(path, model_name, project_name)\n",
+    "with Session(viya4_host, username, password, protocol=\"http\"):\n",
+    "    convert_and_upload_model(path, model_name, project_name)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "40d9c737-f38b-4262-aa4d-e328a4d73c2b",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "dev-py38",
+   "language": "python",
+   "name": "dev-py38"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}