Merge branch 'GoogleCloudPlatform:master' into master

Author: annasochandure-ssk

continuous_training/kubeflow/labs/multiple_frameworks_lab.ipynb

@@ -1022,6 +1022,38 @@
     "!head census_training_pipeline.yaml"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Set the command fields in the pipeline YAML"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!sed -i 's/\\\"command\\\": \\[\\]/\\\"command\\\": \\[python, -u, -m, kfp_component.launcher\\]/g' census_training_pipeline.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!cat census_training_pipeline.yaml | grep \"component.launcher\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You should see 6 lines in the output that were modified by the sed command."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -1064,6 +1096,11 @@
     "* For **Run Type** select **One-Off**\n",
     "* Enter your **Project ID** and hit **Start**. You can now monitor your pipeline run in the UI (it will take about **10 minutes** to complete the run).\n",
     "\n",
+    "\n",
+    "\n",
+    "**NOTE that your pipeline run may fail due to the bug in a BigQuery component that does not handle certain race conditions. If you observe the pipeline failure, submit another pipeline run using the KFP UI using the steps above**\n",
+    "\n",
+    "\n",
     "Now let's set up a recurring run:\n",
     "* Select **Pipelines** then **census_trainer_multiple_models** and click **Create Run**\n",
     "* In the **Experiement** field select **Default**\n",

continuous_training/kubeflow/solutions/multiple_frameworks_kubeflow.ipynb

@@ -1020,6 +1020,38 @@
     "!head census_training_pipeline.yaml"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Set the command fields in the pipeline YAML"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!sed -i 's/\\\"command\\\": \\[\\]/\\\"command\\\": \\[python, -u, -m, kfp_component.launcher\\]/g' census_training_pipeline.yaml"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!cat census_training_pipeline.yaml | grep \"component.launcher\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "You should see 6 lines in the output that were modified by the sed command."
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -1062,6 +1094,10 @@
     "* For **Run Type** select **One-Off**\n",
     "* Enter your **Project ID** and hit **Start**. You can now monitor your pipeline run in the UI (it will take about **10 minutes** to complete the run).\n",
     "\n",
+    "\n",
+    "**NOTE that your pipeline run may fail due to the bug in a BigQuery component that does not handle certain race conditions. If you observe the pipeline failure, submit another pipeline run using the KFP UI using the steps above**\n",
+    "\n",
+    "\n",
     "Now let's set up a recurring run:\n",
     "* Select **Pipelines** then census_trainer_multiple_models and click **Create Run**\n",
     "* In the **Experiement** field select **Default**\n",

model_serving/caip-load-testing/01-prepare-and-deploy.ipynb

@@ -22,7 +22,7 @@
    "source": [
     "## Setup\n",
     "\n",
-    "This Notebook was tested on **AI Platform Notebooks** using the standard TF 2.2 image."
+    "This Notebook was tested on **AI Platform Notebooks** using the standard TF 2.8 image."
    ]
   },
   {
@@ -72,9 +72,7 @@
     "GCS_MODEL_LOCATION = 'gs://{}/models/{}/{}'.format(BUCKET, MODEL_NAME, MODEL_VERSION)\n",
     "THUB_MODEL_HANDLE = 'https://tfhub.dev/google/imagenet/resnet_v2_101/classification/4'\n",
     "IMAGENET_LABELS_URL = 'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt'\n",
-    "IMAGES_FOLDER = 'test_images'\n",
-    "\n",
-    "!gcloud config set project $PROJECT_ID"
+    "IMAGES_FOLDER = 'test_images'"
    ]
   },
   {
@@ -550,7 +548,7 @@
    "source": [
     "!gcloud ai-platform models create {MODEL_NAME} \\\n",
     "  --project {PROJECT_ID} \\\n",
-    "  --regions {REGION}"
+    "  --region {REGION}"
    ]
   },
   {
@@ -559,7 +557,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!gcloud ai-platform models list --project {PROJECT_ID} "
+    "!gcloud ai-platform models list \\\n",
+    "  --project {PROJECT_ID} \\\n",
+    "  --region {REGION}"
    ]
   },
   {
@@ -581,12 +581,13 @@
     "!gcloud beta ai-platform versions create {MODEL_VERSION} \\\n",
     "  --model={MODEL_NAME} \\\n",
     "  --origin={GCS_MODEL_LOCATION} \\\n",
-    "  --runtime-version=2.1 \\\n",
+    "  --runtime-version=2.8 \\\n",
     "  --framework=TENSORFLOW \\\n",
     "  --python-version=3.7 \\\n",
     "  --machine-type={MACHINE_TYPE} \\\n",
     "  --accelerator={ACCELERATOR} \\\n",
-    "  --project={PROJECT_ID}"
+    "  --project={PROJECT_ID} \\\n",
+    "  --region={REGION}"
    ]
   },
   {
@@ -595,7 +596,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "!gcloud ai-platform versions list --model={MODEL_NAME} --project={PROJECT_ID}"
+    "!gcloud ai-platform versions list \\\n",
+    "  --model={MODEL_NAME} --project={PROJECT_ID} --region={REGION}"
    ]
   },
   {
@@ -612,8 +614,14 @@
    "outputs": [],
    "source": [
     "import googleapiclient.discovery\n",
-    "\n",
-    "service = googleapiclient.discovery.build('ml', 'v1')\n",
+    "from google.api_core.client_options import ClientOptions\n",
+    "\n",
+    "prefix = '{}-ml'.format(REGION) if REGION else 'ml'\n",
+    "api_endpoint = 'https://{}.googleapis.com'.format(prefix)\n",
+    "client_options = ClientOptions(api_endpoint=api_endpoint)\n",
+    "service = googleapiclient.discovery.build('ml', 'v1',\n",
+    "                                          cache_discovery=False,\n",
+    "                                          client_options=client_options)\n",
     "name = 'projects/{}/models/{}/versions/{}'.format(PROJECT_ID, MODEL_NAME, MODEL_VERSION)\n",
     "print(\"Service name: {}\".format(name))\n",
     "\n",

model_serving/caip-load-testing/02-perf-testing.ipynb

@@ -19,14 +19,16 @@
    "metadata": {},
    "source": [
     "## Setup\n",
-    "This notebook was tested on **AI Platform Notebooks** using the standard TF 2.2 image."
+    "This notebook was tested on **AI Platform Notebooks** using the standard TF 2.8 image."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Install required packages"
+    "### Install required packages\n",
+    "\n",
+    "You can safely ignore the dependency errors. Confirm the last message starting from \"Successfully installed...\""
    ]
   },
   {
@@ -35,7 +37,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "%pip install -q -U locust google-cloud-monitoring google-cloud-logging google-cloud-monitoring-dashboards"
+     "!pip install --user locust==2.11.1\\\n",
+     "  google-cloud-monitoring==2.11.1\\\n",
+     "  google-cloud-logging==3.2.2\\\n",
+     "  google-cloud-monitoring-dashboards==2.7.2"
    ]
   },
   {
@@ -80,11 +85,11 @@
     "from google.api_core.exceptions import GoogleAPICallError \n",
     "\n",
     "from google.cloud import logging_v2\n",
-    "from google.cloud.logging_v2 import MetricsServiceV2Client\n",
-    "from google.cloud.logging_v2 import LoggingServiceV2Client\n",
+    "from google.cloud.logging_v2.services.metrics_service_v2 import MetricsServiceV2Client\n",
+    "from google.cloud.logging_v2.services.logging_service_v2 import LoggingServiceV2Client\n",
     "\n",
-    "from google.cloud.monitoring_dashboard.v1.types import Dashboard\n",
-    "from google.cloud.monitoring_dashboard.v1 import DashboardsServiceClient\n",
+    "from google.cloud.monitoring_dashboard_v1.types import Dashboard\n",
+    "from google.cloud.monitoring_dashboard_v1 import DashboardsServiceClient\n",
     "from google.cloud.monitoring_v3 import MetricServiceClient\n",
     "from google.cloud.monitoring_v3.query import Query\n",
     "from google.cloud.monitoring_v3.types import TimeInterval\n",
@@ -160,7 +165,7 @@
     "    value_field:str,  \n",
     "    bucket_bounds:List[int]):\n",
     "    \n",
-    "    metric_path = logging_client.metric_path(PROJECT_ID, metric_name)\n",
+    "    metric_path = logging_client.log_metric_path(PROJECT_ID, metric_name)\n",
     "    log_entry_filter = 'resource.type=global AND logName={}'.format(log_path)\n",
     "    \n",
     "    metric_descriptor = {\n",
@@ -203,7 +208,11 @@
     "        logging_client.get_log_metric(metric_path)\n",
     "        print('Metric: {} already exists'.format(metric_path))\n",
     "    except:\n",
-    "        logging_client.create_log_metric(parent, metric)\n",
+    "        request = logging_v2.types.logging_metrics.CreateLogMetricRequest(\n",
+    "            parent=parent,\n",
+    "            metric=metric,\n",
+    "        )\n",
+    "        logging_client.create_log_metric(request)\n",
     "        print('Created metric {}'.format(metric_path))"
    ]
   },
@@ -225,7 +234,7 @@
     "creds , _ = google.auth.default()\n",
     "logging_client = MetricsServiceV2Client(credentials=creds)\n",
     "\n",
-    "parent = logging_client.project_path(PROJECT_ID)\n",
+    "parent = logging_client.common_project_path(PROJECT_ID)\n",
     "log_path = LoggingServiceV2Client.log_path(PROJECT_ID, log_name)"
    ]
   },
@@ -284,12 +293,13 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "metrics = logging_client.list_log_metrics(parent)\n",
+    "request = {'parent': parent}\n",
+    "metrics = logging_client.list_log_metrics(request)\n",
     "\n",
     "if not list(metrics):\n",
     "    print(\"There are not any log based metrics defined in the the project\")\n",
     "else:\n",
-    "    for element in logging_client.list_log_metrics(parent):\n",
+    "    for element in logging_client.list_log_metrics(request):\n",
     "        print(element.metric_descriptor.name)"
    ]
   },
@@ -337,8 +347,12 @@
    "outputs": [],
    "source": [
     "dashboard_proto = Dashboard()\n",
-    "dashboard_proto = ParseDict(dashboard_template, dashboard_proto)\n",
-    "dashboard = dashboard_service_client.create_dashboard(parent, dashboard_proto)"
+    "request = {\n",
+    "    'parent': parent,\n",
+    "    'dashboard': dashboard_proto,\n",
+    "}\n",
+    "dashboard_proto = ParseDict(dashboard_template, dashboard_proto._pb)\n",
+    "dashboard = dashboard_service_client.create_dashboard(request)"
    ]
   },
   {
@@ -347,7 +361,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "for dashboard in dashboard_service_client.list_dashboards(parent):\n",
+    "for dashboard in dashboard_service_client.list_dashboards({'parent': parent}):\n",
     "    print('Dashboard name: {}, Dashboard ID: {}'.format(dashboard.display_name, dashboard.name))"
    ]
   },
@@ -357,7 +371,7 @@
    "source": [
     "## 3. Deploying Locust to a GKE cluster\n",
     "\n",
-    "Before proceeding, you need access to a GKE cluster. The described deployment process can deploy Locust to any GKE cluster as long as there are enough compute resources to support your Locust configuration. The default configuration follows the Locust's best practices and requests one processor core and 4Gi of memory for the Locust master and one processor core and 2Gi of memory for each Locust worker. As you run your tests, it is important to monitor the the master and the workers for resource utilization and fine tune the allocated resources as required.\n",
+    "Before proceeding, you need access to a GKE cluster. You can find a command to create a GKE cluster in [Environment setup](https://github.com/GoogleCloudPlatform/mlops-on-gcp/blob/master/model_serving/caip-load-testing/README.md#environment-setup) section of [README.md](https://github.com/GoogleCloudPlatform/mlops-on-gcp/blob/master/model_serving/caip-load-testing/README.md). The described deployment process can deploy Locust to any GKE cluster as long as there are enough compute resources to support your Locust configuration. The default configuration follows the Locust's best practices and requests one processor core and 4Gi of memory for the Locust master and one processor core and 2Gi of memory for each Locust worker. As you run your tests, it is important to monitor the the master and the workers for resource utilization and fine tune the allocated resources as required.\n",
     "\n",
     "The deployment process has been streamlined using [Kustomize](https://kustomize.io/). As described in the following steps, you can fine tune the baseline configuration by modifying the default `kustomization.yaml` and `patch.yaml` files in the `locust/manifests` folder.\n",
     "\n"
@@ -623,10 +637,10 @@
    "source": [
     "You can try using the following parameter configurations:\n",
     "1. Number of total users to simulate: 152\n",
-    "2. Hatch rate: 1\n",
-    "3. Host: http://ml.googleapis.com\n",
-    "4. Number of users to increase by step: 8\n",
-    "5. Step duration: 1m "
+    "2. Spawn rate: 1\n",
+    "3. Host: `http://[your-region]-ml.googleapis.com`\n",
+    "\n",
+    "**NOTE**: `[your-region]` is the region for deploying the model that you configured as `REGION` in the first notebook. "
    ]
   },
   {