Merge pull request #251622 from cloga/lochen/tsg-code-first

Lochen/tsg code first

Author: PMEds28

articles/machine-learning/prompt-flow/get-started-prompt-flow.md

@@ -26,10 +26,6 @@ This article walks you through the main user journey of using Prompt flow in Azu
 > Prompt flow is **not supported** in the workspace which has data isolation enabled. The enableDataIsolation flag can only be set at the workspace creation phase and can't be updated.
 >
 >Prompt flow is **not supported** in the project workspace which was created with a workspace hub. The workspace hub is a private preview feature.
->
->Prompt flow is **not supported** in workspaces that enable managed VNet. Managed VNet is a private preview feature.
->
->Prompt flow is **not supported** if you secure your Azure AI services account(Azure openAI, Azure cognitive search, Azure content safety) with virtual networks. If you want to use these as connection in prompt flow please allow access from all networks.
 
 In your Azure Machine Learning workspace, you can enable Prompt flow by turning on **Build AI solutions with Prompt flow** in the **Manage preview features** panel.
 

articles/machine-learning/prompt-flow/how-to-create-manage-runtime.md

@@ -108,81 +108,6 @@ Go to runtime detail page and select update button at the top. You can change ne
 > [!NOTE]
 > If you used a custom environment, you need to rebuild it using latest prompt flow image first, and then update your runtime with the new custom environment.
 
-## Troubleshooting guide for runtime
-
-### Common issues
-
-#### My runtime is failed with a system error **runtime not ready** when using a custom environment
-
-:::image type="content" source="./media/how-to-create-manage-runtime/ci-failed-runtime-not-ready.png" alt-text="Screenshot of a failed run on the runtime detail page. " lightbox = "./media/how-to-create-manage-runtime/ci-failed-runtime-not-ready.png":::
-
-First, go to the Compute Instance terminal and run `docker ps` to find the root cause. 
-
-Use  `docker images`  to check if the image was pulled successfully. If your image was pulled successfully, check if the Docker container is running. If it's already running, locate this runtime, which will attempt to restart the runtime and compute instance.
-
-#### Run failed due to "No module named XXX"
-
-This type error usually related to runtime lack required packages. If you're using default environment, make sure image of your runtime is using the latest version, learn more: [runtime update](#update-runtime-from-ui), if you're using custom image and you're using conda environment, make sure you have installed all required packages in your conda environment, learn more: [customize Prompt flow environment](how-to-customize-environment-runtime.md#customize-environment-with-docker-context-for-runtime).
-
-#### Request timeout issue
-
-##### Request timeout error shown in UI
-
-**MIR runtime request timeout error in the UI:**
-
-:::image type="content" source="./media/how-to-create-manage-runtime/mir-runtime-request-timeout.png" alt-text="Screenshot of a MIR runtime timeout error in the studio UI. " lightbox = "./media/how-to-create-manage-runtime/mir-runtime-request-timeout.png":::
-
-Error in the example says "UserError: Upstream request timeout".
-
-**Compute instance runtime request timeout error:**
-
-:::image type="content" source="./media/how-to-create-manage-runtime/ci-runtime-request-timeout.png" alt-text="Screenshot of a compute instance runtime timeout error in the studio UI. " lightbox = "./media/how-to-create-manage-runtime/ci-runtime-request-timeout.png":::
-
-Error in the example says "UserError: Invoking runtime gega-ci timeout, error message: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing".
-
-#### How to identify which node consume the most time
-
-1. Check the runtime logs
-
-2. Trying to find below warning log format
-
-    {node_name} has been running for {duration} seconds.
-
-    For example:
-
-   - Case 1: Python script node running for long time.
-
-        :::image type="content" source="./media/how-to-create-manage-runtime/runtime-timeout-running-for-long-time.png" alt-text="Screenshot of a timeout run logs in the studio UI. " lightbox = "./media/how-to-create-manage-runtime/runtime-timeout-running-for-long-time.png":::
-
-        In this case, you can find that the `PythonScriptNode` was running for a long time (almost 300s), then you can check the node details to see what's the problem.
-
-   - Case 2: LLM node running for long time.
-
-        :::image type="content" source="./media/how-to-create-manage-runtime/runtime-timeout-by-language-model-timeout.png" alt-text="Screenshot of a timeout logs caused by LLM timeout in the studio UI. " lightbox = "./media/how-to-create-manage-runtime/runtime-timeout-by-language-model-timeout.png":::
-
-        In this case, if you find the message `request canceled` in the logs, it may be due to the OpenAI API call taking too long and exceeding the runtime limit.
-
-        An OpenAI API Timeout could be caused by a network issue or a complex request that requires more processing time. For more information, see [OpenAI API Timeout](https://help.openai.com/en/articles/6897186-timeout).
-
-        You can try waiting a few seconds and retrying your request. This usually resolves any network issues.
-
-        If retrying doesn't work, check whether you're using a long context model, such as ‘gpt-4-32k’, and have set a large value for `max_tokens`. If so, it's expected behavior because your prompt may generate a very long response that takes longer than the interactive mode upper threshold. In this situation, we recommend trying 'Bulk test', as this mode doesn't have a timeout setting.
-
-3. If you can't find anything in runtime logs to indicate it's a specific node issue
-
-    Contact the Prompt Flow team ([promptflow-eng](mailto:[email protected])) with the runtime logs. We'll try to identify the root cause.
-
-### Compute instance runtime related
-
-#### How to find the compute instance runtime log for further investigation?
-
-Go to the compute instance terminal and run  `docker logs -<runtime_container_name>`
-
-#### User doesn't have access to this compute instance. Please check if this compute instance is assigned to you and you have access to the workspace. Additionally, verify that you are on the correct network to access this compute instance.
-
-:::image type="content" source="./media/how-to-create-manage-runtime/ci-flow-clone-others.png" alt-text="Screenshot of a don't have access error on the flow page. " lightbox = "./media/how-to-create-manage-runtime/ci-flow-clone-others.png":::
-
-This because you're cloning a flow from others that is using compute instance as runtime. As compute instance runtime is user isolated, you need to create your own compute instance runtime or select a managed online deployment/endpoint runtime, which can be shared with others.
 
 ## Next steps
 

articles/machine-learning/prompt-flow/how-to-customize-environment-runtime.md

@@ -174,87 +174,6 @@ Follow [this document to add custom application](../how-to-create-compute-instan
 
 :::image type="content" source="./media/how-to-customize-environment-runtime/runtime-creation-add-custom-application-ui.png" alt-text="Screenshot of compute showing custom applications. " lightbox = "./media/how-to-customize-environment-runtime/runtime-creation-add-custom-application-ui.png":::
 
-## Create managed online deployment that can be used as Prompt flow runtime (deprecated)
-
-> [!IMPORTANT]
-> Managed online endpoint/deployment as runtime is **deprecated**. Please use [Migrate guide for managed online endpoint/deployment runtime](./migrate-managed-inference-runtime.md).
-
-### Create managed online deployment that can be used as Prompt flow runtime via CLI v2
-
-Learn more about [deploy and score a machine learning model by using an online endpoint](../how-to-deploy-online-endpoints.md)
-
-#### Create managed online endpoint
-
-To define a managed online endpoint, you can use the following yaml template. Make sure to replace the `ENDPOINT_NAME` with the desired name for your endpoint.
-
-```yaml
-$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
-name: <ENDPOINT_NAME>
-description: this is a sample promptflow endpoint
-auth_mode: key
-```
-
-Use following CLI command `az ml online-endpoint create -f <yaml_file> -g <resource_group> -w <workspace_name>` to create managed online endpoint. To learn more, see [Deploy and score a machine learning model by using an online endpoint](../how-to-deploy-online-endpoints.md).
-
-#### Create Prompt flow runtime image config file
-
-To configure your Prompt flow runtime, place the following config file in your model folder. This config file provides the necessary information for the runtime to work properly.
-
-For the `mt_service_endpoint` parameter, follow this format: `https://<region>.api.azureml.ms`. For example, if your region is eastus, then your service endpoint should be `https://eastus.api.azureml.ms`
-
-```yaml
-storage:
-  storage_account: <WORKSPACE_LINKED_STORAGE>
-deployment:
-  subscription_id: <SUB_ID>
-  resource_group: <RG_NAME>
-  workspace_name: <WORKSPACE_NAME>
-  endpoint_name: <ENDPOINT_NAME>
-  deployment_name: blue
-  mt_service_endpoint: <PROMPT_FLOW_SERVICE_ENDPOINT>
-```
-
-#### Create managed online endpoint
-
-You need to replace the following placeholders with your own values:
-
-- `ENDPOINT_NAME`: the name of the endpoint you created in the previous step
-- `PRT_CONFIG_FILE`: the name of the config file that contains the port and runtime settings. Include the parent model folder name, for example, if model folder name is `model`, then the config file name should be `model/config.yaml`.
-- `IMAGE_NAME` to name of your own image, for example: `mcr.microsoft.com/azureml/promptflow/promptflow-runtime:<newest_version>`, you can also follow [Customize environment with docker context for runtime](#customize-environment-with-docker-context-for-runtime) to create your own environment.
-
-```yaml
-$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
-name: blue
-endpoint_name: <ENDPOINT_NAME>
-type: managed
-model:
-  path: ./
-  type: custom_model
-instance_count: 1
-# 4core, 32GB
-instance_type: Standard_E4s_v3
-request_settings:
-  max_concurrent_requests_per_instance: 10
-  request_timeout_ms: 90000
-environment_variables:
-  PRT_CONFIG_FILE: <PRT_CONFIG_FILE>
-environment:
-  name: promptflow-runtime
-  image: <IMAGE_NAME>
-  inference_config:
-    liveness_route:
-      port: 8080
-      path: /health
-    readiness_route:
-      port: 8080
-      path: /health
-    scoring_route:
-      port: 8080
-      path: /score
-
-```
-
-Use following CLI command `az ml online-deployment create -f <yaml_file> -g <resource_group> -w <workspace_name>` to create managed online deployment that can be used as a Prompt flow runtime.
 
 ## Next steps
 

articles/machine-learning/prompt-flow/how-to-deploy-for-real-time-inference.md

@@ -45,6 +45,12 @@ If you didn't complete the tutorial, you need to build a flow. Testing the flow
 
 We'll use the sample flow **Web Classification** as example to show how to deploy the flow. This sample flow is a standard flow. Deploying chat flows is similar. Evaluation flow doesn't support deployment.
 
+## Define the environment used by deployment
+
+When you deploy prompt flow to managed online endpoint in UI. You need define the environment used by this flow. By default, it will use the latest prompt image version. You can specify extra packages you needed in `requirements.txt`. You can find `requirements.txt` in the root folder of your flow folder, which is system generated file.
+
+:::image type="content" source="./media/how-to-deploy-for-real-time-inference/requirements-text.png" alt-text="Screenshot of Web requirements-text. " lightbox = "./media/how-to-deploy-for-real-time-inference/requirements-text.png":::
+
 ## Create an online endpoint
 
 Now that you have built a flow and tested it properly, it's time to create your online endpoint for real-time inference. 

articles/machine-learning/prompt-flow/how-to-secure-prompt-flow.md

@@ -54,27 +54,25 @@ Workspace managed virtual network is the recommended way to support network isol
 
     :::image type="content" source="./media/how-to-secure-prompt-flow/outbound-rule-non-azure-resources.png" alt-text="Screenshot of user defined outbound rule for non Azure resource." lightbox = "./media/how-to-secure-prompt-flow/outbound-rule-non-azure-resources.png":::
 
+4. In workspace which enable managed VNet, you can only deploy prompt flow to managed online endpoint. You can follow [Secure your managed online endpoints with network isolation](../how-to-secure-kubernetes-inferencing-environment.md) to secure your managed online endpoint.
+
 ## Secure prompt flow use your own virtual network
 
 - To set up Azure Machine Learning related resources as private, see [Secure workspace resources](../how-to-secure-workspace-vnet.md).
 - Meanwhile, you can follow [private Azure Cognitive Services](../../ai-services/cognitive-services-virtual-networks.md) to make them as private.
+- If you want to deploy prompt flow in workspace which secured by your own virtual network, you can deploy it to AKS cluster which is in the same virtual network. You can follow [Secure your RAG workflows with network isolation](../how-to-secure-rag-workflows.md) to secure your AKS cluster.
 - You can either create private endpoint to the same virtual network or leverage virtual network peering to make them communicate with each other.
 
-## Limitations
+## Known limitations
 
-- Only public access enable storage account is supported. You can't use private storage account now.
+- Only public access enable storage account is supported. You can't use private storage account now. Find workaround here: [Why I can't create or upgrade my flow when I disable public network access of storage account?](./tools-reference/troubleshoot-guidance.md#why-i-cant-create-or-upgrade-my-flow-when-i-disable-public-network-access-of-storage-account)
 - Workspace hub / lean workspace and AI studio don't support bring your own virtual network.
 - Managed online endpoint only supports workspace managed virtual network. If you want to use your own virtual network, you may need one workspace for prompt flow authoring with your virtual network and another workspace for prompt flow deployment using managed online endpoint with workspace managed virtual network.
 
-## FAQ
-
-### Why I can't create or upgrade my flow when I disable public network access of storage account?
-Prompt flow rely on fileshare to store snapshot of flow. Prompt flow didn't support private storage account now. Here are some workarounds you can try:
-- Make the storage account as public access enabled if there is no security concern. 
-- If you are only use UI to authoring promptflow, you can add following flights (flight=PromptFlowCodeFirst=false) to use our old UI.
-- You can use our CLI/SDK to authoring promptflow, CLI/SDK authong didn't rely on fileshare. See [Integrate Prompt Flow with LLM-based application DevOps ](how-to-integrate-with-llm-app-devops.md).
-
 ## Next steps
 
 - [Secure workspace resources](../how-to-secure-workspace-vnet.md)
-- [Workspace managed network isolation](../how-to-managed-network.md)
+- [Workspace managed network isolation](../how-to-managed-network.md)
+- [Secure Azure Kubernetes Service inferencing environment](../how-to-secure-online-endpoint.md)
+- [Secure your managed online endpoints with network isolation](../how-to-secure-kubernetes-inferencing-environment.md)
+- [Secure your RAG workflows with network isolation](../how-to-secure-rag-workflows.md)