llms.txt

# Alauda AI / Machine Learning Documentation

> Documentation for Alauda's AI and machine learning platform, including Kubeflow, KServe, KubeRay, LWS, Kueue, Envoy AI Gateway, Dify, Feast, Llama Stack, Label Studio, TrustyAI, LLM Compressor, model inference, infrastructure management, and related installation, upgrade, monitoring, and learning guides.

## administer

- [docs/en/administer/hardware_profile/cpu_and_gpu_profiles.mdx](docs/en/administer/hardware_profile/cpu_and_gpu_profiles.mdx): How-to that contrasts a CPU-only HardwareProfile (cpu and memory identifiers only, generic worker nodeSelector) with a GPU-accelerated profile that adds an nvidia.com/gpu Accelerator identifier, an accelerator=nvidia-t4 nodeSelector, and an nvidia.com/gpu:NoSchedule toleration. Provides full infrastructure.opendatahub.io/v1alpha1 HardwareProfile YAML for both standard-cpu-profile and gpu-t4-profile so administrators can isolate scikit-learn/XGBoost-style CPU inference from LLM workloads that need physical GPU acceleration.
- [docs/en/administer/hardware_profile/create_hardware_profile_cli.mdx](docs/en/administer/hardware_profile/create_hardware_profile_cli.mdx): Walks through creating a HardwareProfile from the command line by applying a sample gpu-high-performance-profile YAML (apiVersion infrastructure.opendatahub.io/v1alpha1) into the kube-public namespace with kubectl apply, then verifying with kubectl get hardwareprofile. The sample profile defines nvidia.com/gpu, cpu, and memory identifiers with min/max/default counts plus an accelerator=nvidia-a100 nodeSelector and an nvidia.com/gpu:NoSchedule toleration so data scientists can pick "GPU High Performance" in the Inference Service UI.
- [docs/en/administer/hardware_profile/hardware_profile.mdx](docs/en/administer/hardware_profile/hardware_profile.mdx): Step-by-step administrator guide for managing HardwareProfile resources through the Alauda AI UI, covering creation, update, and deletion as well as how data scientists consume a profile when deploying InferenceService or LLMInferenceService workloads from the Service Manage page. Walks through configuring built-in and custom resource identifiers (cpu, memory, nvidia.com/gpu, custom accelerator/Other types) with default/min/max bounds, plus node selectors and tolerations, and explains the Hardware Profile vs. Custom Config Type choice and View Detail / Custom Configuration controls in the deployment form.
- [docs/en/administer/hardware_profile/index.mdx](docs/en/administer/hardware_profile/index.mdx): Top-level Hardware Profile section landing page that renders an Overview component aggregating the Introduction, admin function guides, and how-to articles (CLI creation, CPU vs GPU profiles, scheduling to specific GPU nodes) for governing Alauda AI inference resource allocation.
- [docs/en/administer/hardware_profile/intro.mdx](docs/en/administer/hardware_profile/intro.mdx): Conceptual introduction explaining why Alauda AI ships HardwareProfile as an abstraction over raw Kubernetes scheduling for InferenceService and LLMInferenceService workloads. Covers the four pillars of the design: topology and accelerator abstraction via embedded node selectors and tolerations, dynamic bounded customization with Minimum/Default/Maximum resource limits, a dedicated Mutating Webhook that injects constraints and auto-corrects request/limit mismatches, and native interoperability with custom serving engines so the profile reaches the active predictor container.
- [docs/en/administer/hardware_profile/schedule_to_specific_gpu_nodes.mdx](docs/en/administer/hardware_profile/schedule_to_specific_gpu_nodes.mdx): Explains how to pin AI inference workloads to dedicated GPU hardware by combining Node Selectors (for example accelerator=nvidia-a100 or nvidia.com/gpu.present=true) with Tolerations that match taints like nvidia.com/gpu:NoSchedule inside a Hardware Profile. Demonstrates the recommended Key/Operator/Effect (Exists, NoSchedule) configuration so that selecting the profile from the UI transparently targets A100/H100-class nodes without data scientists hand-editing pod specs.
- [docs/en/administer/index.mdx](docs/en/administer/index.mdx): Chapter landing page for platform administrators, covering hardware configuration, namespace and multi-tenant management, and cross-component administration such as MLflow workspace permissions and the Kagenti secure profile. Points to MLflow under Develop and Kagenti under Build AI Applications, and to Train for training quota governance.
- [docs/en/administer/multi_tenant/index.mdx](docs/en/administer/multi_tenant/index.mdx): Landing page for the Connections how-to guides, rendering an overview component that links to task-based tutorials for creating, updating, and consuming Connection resources for external model and data sources in Alauda AI projects.
- [docs/en/administer/multi_tenant/mlflow_workspaces.mdx](docs/en/administer/multi_tenant/mlflow_workspaces.mdx): Guide to MLflow multi-tenancy on Alauda AI, where each workspace is a Kubernetes namespace and access to experiments, runs, datasets, and registered models is authorized by Kubernetes RBAC. Covers exposing a namespace as a workspace, granting user access, selecting a workspace from a client, and troubleshooting.
- [docs/en/administer/multi_tenant/namespace-manage.mdx](docs/en/administer/multi_tenant/namespace-manage.mdx): Procedure for onboarding a Kubernetes namespace as an Alauda AI tenant: creating the namespace under a project in the Web Console with Pod Security Policies set to privileged (otherwise AI Inference workloads will refuse to run), assigning Alauda AI Editor/Owner/Viewer roles to users through User Role Management, and finally registering it under Admin > Namespace Manage > Management Namespace.
- [docs/en/administer/secure_profile/enable_secure_profile.mdx](docs/en/administer/secure_profile/enable_secure_profile.mdx): Procedure for turning on the Kagenti Operator secure profile, which adds SPIRE-based zero-trust workload identity, Keycloak OAuth2/JWT authentication, and an Istio ambient mesh enforced by an injected AuthBridge sidecar. Covers enabling it on the operand, opting namespaces in, deploying a secured agent, and verifying sidecar injection, SVID delivery, and Keycloak client registration.
- [docs/en/administer/secure_profile/index.mdx](docs/en/administer/secure_profile/index.mdx): Section index for the Kagenti secure profile, linking the dependency installation guide and the procedure that enables the profile on a cluster.
- [docs/en/administer/secure_profile/install_secure_dependencies.mdx](docs/en/administer/secure_profile/install_secure_dependencies.mdx): Installs the cluster components the Kagenti secure profile depends on: cert-manager, SPIRE, Keycloak, and Istio in ambient mode. Describes the common platform package installation pattern, the SPIRE server, agent, CSI driver, and trust bundle, Keycloak operator and instance deployment with realm import, and Istio ambient setup.

## ai_applications

- [docs/en/ai_applications/components/dify/features.mdx](docs/en/ai_applications/components/dify/features.mdx): Concise catalog of Dify capabilities organized into LLM application development (Assistant, Text Generator, Agent, Workflow/Chatflow; multi-provider LLM integration; visual prompt management), RAG (knowledge base ingestion, pgvector backing in the Helm chart, hybrid search and re-ranking, parent-child context, dataset versioning), Workflow & Agent (visual node-based editor, tool use, error handling, observability), and API & Integration (REST Service API, per-app API keys, Web App publish/embed, SDKs).
- [docs/en/ai_applications/components/dify/index.mdx](docs/en/ai_applications/components/dify/index.mdx): Component landing page for Dify in the Build AI Applications chapter, linking its introduction, installation guide, and feature reference.
- [docs/en/ai_applications/components/dify/install.mdx](docs/en/ai_applications/components/dify/install.mdx): Step-by-step installation of Dify on Alauda Container Platform using the helm-based Dify OLM Operator. Covers publishing the operator package, installing the operator, preparing external PostgreSQL, Redis, optional pgvector, and RWX or S3 storage, choosing an access method and gateway, providing plugin-install proxies for restricted networks, and creating the Dify instance and connection secrets.
- [docs/en/ai_applications/components/dify/intro.mdx](docs/en/ai_applications/components/dify/intro.mdx): Introduction to Dify, an open-source LLM application development platform offering a visual interface for AI workflows, RAG pipelines, agent capabilities, and model management. Describes core concepts and use cases from simple chatbots and text generators to multi-step agent workflows, and links the official documentation.
- [docs/en/ai_applications/components/index.mdx](docs/en/ai_applications/components/index.mdx): Section index for the components the Build AI Applications chapter relies on, gathering the introduction and installation guides for Dify, Llama Stack, Kagenti, and the MCP Lifecycle Operator.
- [docs/en/ai_applications/components/kagenti/index.mdx](docs/en/ai_applications/components/kagenti/index.mdx): Component landing page for the Kagenti Operator, linking its introduction, installation guide, and security architecture reference.
- [docs/en/ai_applications/components/kagenti/install.mdx](docs/en/ai_applications/components/kagenti/install.mdx): Installs the Kagenti Operator as an OLM operator from the platform OperatorHub. Lists prerequisites including ACP v4.0 or later, amd64 or arm64 clusters, OLM, and cert-manager for the admission webhook certificate; covers uploading and installing the operator, creating the Kagenti operand, verifying the CSV, controllers, and agent CRDs, and contrasts the core and secure profiles.
- [docs/en/ai_applications/components/kagenti/intro.mdx](docs/en/ai_applications/components/kagenti/intro.mdx): Introduction to the Kagenti Operator, a Kubernetes-native operator that enrolls, discovers, and secures AI agents and MCP tools running as ordinary workloads. Describes the `agent.kagenti.dev` CRDs, key features, the default core profile with agent-to-agent discovery, and typical use cases for turning a plain Deployment into a managed agent.
- [docs/en/ai_applications/components/kagenti/security_architecture.mdx](docs/en/ai_applications/components/kagenti/security_architecture.mdx): Reference for the Kagenti Operator zero-trust security stack: SPIFFE/SPIRE workload identity, Istio ambient mesh transport security, and OPA plus AuthBridge plus Keycloak authorization with an example global Rego policy. Documents the AuthBridge sidecar, the AgentRuntime security fields, and how to enable the opt-in secure profile.
- [docs/en/ai_applications/components/llama_stack/features.mdx](docs/en/ai_applications/components/llama_stack/features.mdx): Enumerates Llama Stack's main capabilities: a centralized server hosting inference/agents/safety/tool runtime/vector I/O/files, remote and inline providers (meta-reference, sqlite-vec, localfs), Kubernetes deployment via `LlamaStackDistribution`, `@client_tool` decorator-based agent creation with streaming sessions, YAML stack configuration with `${env.VAR:~default}` fallbacks, multiple distributions (starter, postgres-demo, meta-reference-gpu), and the `llama-stack-client` Python 3.12+ SDK including PGVector-backed vector store APIs.
- [docs/en/ai_applications/components/llama_stack/index.mdx](docs/en/ai_applications/components/llama_stack/index.mdx): Component landing page for Alauda Build of Llama Stack, linking its introduction, installation guide, feature reference, and quickstart.
- [docs/en/ai_applications/components/llama_stack/install.mdx](docs/en/ai_applications/components/llama_stack/install.mdx): Installs Alauda Build of Llama Stack by uploading the Llama Stack Operator package and setting `spec.components.llamaStack.managementState` to `Managed` in the `default` `AmlCluster`; the operator registers the `LlamaStackDistribution` CRD. Then deploys a Llama Stack Server through a `LlamaStackDistribution` custom resource (apiVersion `llamastack.io/v1alpha1`), covering required `VLLM_URL` configuration pointing at a vLLM OpenAI-compatible endpoint, optional `VLLM_API_TOKEN` Secret, PostgreSQL persistence, PGVector-backed vector stores via ACP PostgreSQL, Milvus remote vector stores, Hugging Face mirror/offline embedding-model caching (`HF_ENDPOINT`, `HF_HUB_OFFLINE`), and vLLM predictor tool-calling flags (`--enable-auto-tool-choice`, `--tool-call-parser hermes`) needed for agent flows that use client-side or MCP tools.
- [docs/en/ai_applications/components/llama_stack/intro.mdx](docs/en/ai_applications/components/llama_stack/intro.mdx): Introduces Llama Stack as a framework for building AI agents with tools, explaining its core building blocks: the Llama Stack Server deployed via Operator on Kubernetes, the `llama-stack-client` Python SDK, agents and `@client_tool`-decorated tools, YAML configuration registering providers (inference, agents, safety, vector_io, files) and models such as DeepSeek over OpenAI-compatible APIs, and links to upstream documentation at llamastack.github.io.
- [docs/en/ai_applications/components/llama_stack/quickstart.mdx](docs/en/ai_applications/components/llama_stack/quickstart.mdx): Walks through running the `llama-stack_quickstart.ipynb` notebook against a deployed Llama Stack Server, demonstrating both `@client_tool` client-side tools and FastMCP-based MCP tools registered with `toolgroups.register`, plus an optional PGVector flow that uploads a file via `client.files.create`, creates a `provider_id="pgvector"` vector store, and runs hybrid search with `search_mode="hybrid"`; also includes an FAQ for installing a Python 3.12 ipykernel from `python-build-standalone` so notebooks can use `llama-stack-client==0.6.0`.
- [docs/en/ai_applications/components/mcp_lifecycle_operator/index.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/index.mdx): Component landing page for Alauda Build of MCP Lifecycle Operator, linking its introduction, installation guide, and quickstart.
- [docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/install.mdx): Installs Alauda Build of MCP Lifecycle Operator as an OLM operator from the platform OperatorHub. Covers uploading the operator package, installing the operator, verifying that the CSV reports Succeeded, the controller pod is Running, and the MCPServer CRD is registered, and upgrading the operator.
- [docs/en/ai_applications/components/mcp_lifecycle_operator/intro.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/intro.mdx): Introduction to Alauda Build of MCP Lifecycle Operator, based on the Kubernetes SIG Apps MCP Lifecycle Operator. Describes the `MCPServer` custom resource in `mcp.x-k8s.io/v1alpha1` that declares an MCP server image, port, endpoint path, environment, configuration mounts, resources, and security settings, plus key features, how it works, and use cases.
- [docs/en/ai_applications/components/mcp_lifecycle_operator/quickstart.mdx](docs/en/ai_applications/components/mcp_lifecycle_operator/quickstart.mdx): Quickstart that creates an `MCPServer` resource and verifies that the operator produces the backing Deployment and Service. Covers prerequisites, creating and verifying the resource, testing the MCP endpoint, updating the port or path, common configuration options, and cleanup.
- [docs/en/ai_applications/demo_advanced_secure_profile.mdx](docs/en/ai_applications/demo_advanced_secure_profile.mdx): Advanced variant of the weather agent demo in which AuthBridge is injected on both the agent and the tool, and every agent-to-tool call performs an OAuth 2.0 RFC 8693 token exchange so the tool receives an audience-scoped token. Covers the ConfigMaps involved, the Keycloak client-id convention, deploying the tool with a sidecar, configuring token exchange, preparing outbound routes, and deploying the secured agent.
- [docs/en/ai_applications/demo_with_secure_profile.mdx](docs/en/ai_applications/demo_with_secure_profile.mdx): Runs the weather agent scenario under the Kagenti secure profile: the AuthBridge sidecar is injected into the agent, the operator registers a per-workload Keycloak client, SPIRE issues a workload SVID, and every request must carry a valid Bearer token. Covers deploying the MCP tool server, deploying the secured agent, and verifying sidecar injection, SVID delivery, and token enforcement.
- [docs/en/ai_applications/deploy_agent_with_agentruntime.mdx](docs/en/ai_applications/deploy_agent_with_agentruntime.mdx): Smallest end-to-end deployment on the Kagenti Operator core profile, with agent enrollment and dynamic discovery but no identity sidecars, Keycloak, or mesh mTLS. Builds a weather agent that calls a weather MCP tool, covering how to find the model endpoint and name, deploying the tool server and the agent, and checking `AgentRuntime` status.
- [docs/en/ai_applications/envoy_ai_gateway_mcp.mdx](docs/en/ai_applications/envoy_ai_gateway_mcp.mdx): Example that deploys the Rosso weather and restaurant-reservation MCP servers, aggregates them behind a single Envoy Gateway data plane, and connects a Kagenti generic agent to one unified `/mcp` endpoint. Covers validated versions, the topology and HTTPRoute count, source manifests and images, testing the unified endpoint, and cleanup.
- [docs/en/ai_applications/index.mdx](docs/en/ai_applications/index.mdx): Chapter landing page for building AI applications, covering agent runtimes, MCP servers, and RAG applications, with the components the chapter relies on listed under Components. Notes that enabling the secure profile and installing its dependencies are administrator tasks covered in the Administer chapter.

## apis

- [docs/en/apis/index.mdx](docs/en/apis/index.mdx): Top-level landing page for the Alauda AI API Reference section, rendering an overview component that links to the platform's Kubernetes APIs and ACP-style API listings. Acts as the entry point under the API Reference weight slot in the docs navigation.
- [docs/en/apis/intro.mdx](docs/en/apis/intro.mdx): Introduction page for the Alauda AI API surface, embedding the `<AcpApisOverview />` component that auto-generates a catalog of the ACP REST APIs exposed by the platform. Serves as the first stop for readers exploring HTTP-style API endpoints rather than CRDs.
- [docs/en/apis/kubernetes_apis/index.mdx](docs/en/apis/kubernetes_apis/index.mdx): Landing page for the Kubernetes APIs section, listing the Custom Resource groups Alauda AI exposes (Workbench/Kubeflow workspaces, manage.aml.dev namespaces, amlclusters.aml.dev operator resources, and KServe serving). Renders an overview component that aggregates child CRD reference pages.
- [docs/en/apis/kubernetes_apis/kubeflow.org/index.mdx](docs/en/apis/kubernetes_apis/kubeflow.org/index.mdx): Section index for the Kubeflow-derived Workbench APIs, grouping the `kubeflow.org/v1beta1` Workspace and WorkspaceKind CRDs that back Alauda AI's notebook/IDE workbench feature. Provides the entry point for readers configuring Jupyter-style developer workspaces.
- [docs/en/apis/kubernetes_apis/kubeflow.org/workspacekinds.mdx](docs/en/apis/kubernetes_apis/kubeflow.org/workspacekinds.mdx): Generated CRD reference for `WorkspaceKind` in `kubeflow.org/v1beta1`, rendered via the `<K8sCrd>` component. Documents the cluster-scoped template that defines image options, pod overrides, and lifecycle settings shared by individual user Workspaces in the Kubeflow Notebooks 2.0 / Workbench experience.
- [docs/en/apis/kubernetes_apis/kubeflow.org/workspaces.mdx](docs/en/apis/kubernetes_apis/kubeflow.org/workspaces.mdx): Generated CRD reference for the `Workspace` resource in `kubeflow.org/v1beta1`, rendered via `<K8sCrd>`. Documents the per-user notebook workspace instance that references a WorkspaceKind and drives the running IDE pod, PVCs, and connection bindings inside Alauda AI's Workbench.
- [docs/en/apis/kubernetes_apis/manage/amlnamespaces.mdx](docs/en/apis/kubernetes_apis/manage/amlnamespaces.mdx): Generated CRD reference for `AmlNamespace` in `manage.aml.dev/v1alpha1`, the Alauda AI-managed namespace resource that projects use to declare project-level configuration, quotas, and platform metadata layered on top of a Kubernetes namespace.
- [docs/en/apis/kubernetes_apis/manage/index.mdx](docs/en/apis/kubernetes_apis/manage/index.mdx): Section index for the `manage.aml.dev` API group, anchoring CRDs such as `AmlNamespace` that the management plane uses for project/namespace lifecycle. Renders an overview component listing the manage-group resources documented in this site.
- [docs/en/apis/kubernetes_apis/operator/amlclusters.mdx](docs/en/apis/kubernetes_apis/operator/amlclusters.mdx): Generated CRD reference for `AmlCluster` in `amlclusters.aml.dev/v1alpha1`, the top-level operator resource that installs and reconciles an Alauda AI deployment on a Kubernetes cluster, including its component versions and runtime configuration.
- [docs/en/apis/kubernetes_apis/operator/index.mdx](docs/en/apis/kubernetes_apis/operator/index.mdx): Section index for the Alauda AI Operator APIs, gathering the `amlclusters.aml.dev` CRDs (notably `AmlCluster`) that the operator uses to install and manage the platform. Entry point for cluster administrators authoring operator-level resources.
- [docs/en/apis/kubernetes_apis/serving.kserve.io/clusterservingruntimes.mdx](docs/en/apis/kubernetes_apis/serving.kserve.io/clusterservingruntimes.mdx): Generated CRD reference for KServe's `ClusterServingRuntime` in `serving.kserve.io/v1alpha1`, rendered via `<K8sCrd>`. Documents the cluster-scoped runtime template (container image, supported model formats, predictor protocol) that `InferenceService` resources select when serving a model.
- [docs/en/apis/kubernetes_apis/serving.kserve.io/index.mdx](docs/en/apis/kubernetes_apis/serving.kserve.io/index.mdx): Section index for the KServe Inference Service APIs in the `serving.kserve.io` group, covering both `InferenceService` (v1beta1) and `ClusterServingRuntime` (v1alpha1) CRDs that drive model deployment and serving in Alauda AI.
- [docs/en/apis/kubernetes_apis/serving.kserve.io/inferenceservices.mdx](docs/en/apis/kubernetes_apis/serving.kserve.io/inferenceservices.mdx): Generated CRD reference for KServe's `InferenceService` in `serving.kserve.io/v1beta1`, rendered via `<K8sCrd>`. Describes the primary resource used to deploy a model in Alauda AI - including predictor/transformer/explainer specs, storage URI, runtime selection, and autoscaling - which the platform's model-deployment UI ultimately produces.

## deploy

- [docs/en/deploy/components/envoy_ai_gateway/index.mdx](docs/en/deploy/components/envoy_ai_gateway/index.mdx): Component landing page for Alauda Build of Envoy AI Gateway, linking its introduction and installation guide.
- [docs/en/deploy/components/envoy_ai_gateway/intro.mdx](docs/en/deploy/components/envoy_ai_gateway/intro.mdx): Introduction to Alauda Build of Envoy AI Gateway, a Kubernetes-native AI gateway layer built on Envoy Gateway that provides AI-aware routing by model name and backend availability, an OpenAI-compatible API, and policy enforcement for inference workloads. Links the gateway guides and upstream documentation. Managed by Alauda AI through the `envoyAIGateway` component of the `default` `AmlCluster`.
- [docs/en/deploy/components/index.mdx](docs/en/deploy/components/index.mdx): Section index for the components the Deploy chapter relies on, gathering the introduction and installation guides for KServe, Envoy AI Gateway, LeaderWorkerSet, and InferNex Bridge.
- [docs/en/deploy/components/infernex_bridge/index.mdx](docs/en/deploy/components/infernex_bridge/index.mdx): Component landing page for Alauda Build of InferNex Bridge, linking its introduction and installation guide.
- [docs/en/deploy/components/infernex_bridge/install.mdx](docs/en/deploy/components/infernex_bridge/install.mdx): Installs Alauda Build of InferNex Bridge v26.6.0 from the OLM alpha channel on arm64 clusters. Covers required platform and inference dependencies, the CRDs the operator installs, runtime templates and images, optional dependencies including EagleEye observability, uploading and installing the operator, verification, community examples, and the upgrade and rollback procedures.
- [docs/en/deploy/components/infernex_bridge/intro.mdx](docs/en/deploy/components/infernex_bridge/intro.mdx): Introduction to Alauda Build of InferNex Bridge, based on the openFuyao InferNex project. Connects KServe `LLMInferenceService` workloads with the InferNex inference acceleration stack and provides native `InferNexService` APIs for clusters that do not use KServe. Describes both deployment modes, supported inference engine topologies, and capabilities.
- [docs/en/deploy/components/kserve/index.mdx](docs/en/deploy/components/kserve/index.mdx): Component landing page for Alauda Build of KServe, linking its introduction and installation guide.
- [docs/en/deploy/components/kserve/intro.mdx](docs/en/deploy/components/kserve/intro.mdx): Introduces **Alauda Build of KServe** as Alauda's distribution of upstream KServe, splitting coverage into Generative AI features (llm-d distributed inference with KV-cache-aware scheduling and Leader/Worker multi-node parallelism, vLLM runtime with PagedAttention, OpenAI-compatible `/chat/completions` streaming, autoscaling including scale-to-zero, and Envoy Gateway + GIE integration) and Predictive AI features (the `InferenceService` CRD with canary rollouts, pre-integrated runtimes like TensorFlow Serving, TorchServe, Triton, SKLearn, XGBoost, `ClusterServingRuntime`/`ServingRuntime` for custom runtimes, and `InferenceGraph` for pipeline composition). Includes upstream documentation links for KServe, llm-d, LeaderWorkerSet, Envoy Gateway, Envoy AI Gateway, and Gateway API Inference Extension. Managed by Alauda AI through the `kserve` component of the `default` `AmlCluster`; see Install Alauda AI for enabling it and the KServe service configuration.
- [docs/en/deploy/components/lws/index.mdx](docs/en/deploy/components/lws/index.mdx): Component landing page for Alauda Build of LeaderWorkerSet, linking its introduction and installation guide.
- [docs/en/deploy/components/lws/intro.mdx](docs/en/deploy/components/lws/intro.mdx): Introduces Alauda Build of LeaderWorkerSet, a packaging of the upstream Kubernetes SIG `lws` project that provides a `LeaderWorkerSet` CRD for deploying groups of one leader and N worker pods as a unit; highlights co-scheduling with topology spread constraints for NVLink/InfiniBand interconnects, multi-node LLM inference for models like Llama 3.1 405B using tensor/pipeline parallelism (required by Alauda Build of KServe), distributed training with PyTorch DDP/DeepSpeed/Megatron-LM, group-level rolling updates and failure recovery, and leader-first startup sequencing. Managed by Alauda AI through the `lws` component of the `default` `AmlCluster`.
- [docs/en/deploy/gateway/cost_management_chargeback.mdx](docs/en/deploy/gateway/cost_management_chargeback.mdx): How-to guide that defines a custom cost model for AI Gateway token usage and turns it into per-namespace bills through Alauda Cost Management. Adds a collection configuration on the agent cluster, a display and storage configuration on the server cluster, and a price in the cost model, consuming the OpenTelemetry GenAI token metric keyed on the `user_namespace` label.
- [docs/en/deploy/gateway/external_provider_routing.mdx](docs/en/deploy/gateway/external_provider_routing.mdx): How-to guide for fronting external LLM providers such as OpenAI, AWS Bedrock, Azure OpenAI, GCP Vertex AI, and Anthropic behind one OpenAI-compatible endpoint. Injects upstream credentials with a `BackendSecurityPolicy`, routes by model name, and fails over between providers so consumers never hold provider keys and the gateway becomes the controlled egress point.
- [docs/en/deploy/gateway/identity_authentication.mdx](docs/en/deploy/gateway/identity_authentication.mdx): How-to guide for authenticating inference requests at the Envoy AI Gateway edge with an Envoy Gateway `SecurityPolicy` attached to the `HTTPRoute` generated by an `AIGatewayRoute`. Copies selected claims into request headers that token quotas and usage metering consume as the per-tenant key, turning an SSO token or API key into an identity for multi-tenant policy.
- [docs/en/deploy/gateway/index.mdx](docs/en/deploy/gateway/index.mdx): Section index for the inference gateway how-to guides covering consumer authentication, token quotas, usage metering, external provider routing, and token chargeback.
- [docs/en/deploy/gateway/token_rate_limiting.mdx](docs/en/deploy/gateway/token_rate_limiting.mdx): How-to guide for rate limiting by token usage rather than request count, with a separate budget per caller identity. Uses `AIGatewayRoute.llmRequestCosts` to extract token counts into Envoy dynamic metadata and a Redis global rate limit backend to accumulate cost, enabling per-user, per-department, and per-tier token quotas on one gateway.
- [docs/en/deploy/gateway/usage_metering.mdx](docs/en/deploy/gateway/usage_metering.mdx): How-to guide for collecting Envoy AI Gateway token metrics that follow OpenTelemetry GenAI semantic conventions, labelled with caller identity, into the platform monitoring stack. Covers the pipeline from gateway metric emission through identity labelling, `PodMonitor` collection, and `MonitorDashboard` presentation, feeding chargeback in Alauda Cost Management.
- [docs/en/deploy/index.mdx](docs/en/deploy/index.mdx): Chapter landing page for deployment and inference, covering creating and scaling inference services, model management and storage, Model as a Service, quota and metering at the inference gateway, and model compression, with the chapter's components listed under Components.
- [docs/en/deploy/inference_service/guides/accurately_schedule.mdx](docs/en/deploy/inference_service/guides/accurately_schedule.mdx): Guide to scheduling KServe `InferenceService` pods onto GPU nodes with compatible CUDA driver versions, solving runtime-vs-driver mismatches that the default Kubernetes scheduler ignores. Operators tag each GPU node with `nvidia.com/cuda.runtime.major` / `.minor` labels (manually via `nvidia-smi` + `kubectl label`, or automatically via the Node Feature Discovery plugin's GFD extension), then attach a `preferredDuringSchedulingIgnoredDuringExecution` `nodeAffinity` block matching the `cpaas.io/cuda-version` on the chosen `ClusterServingRuntime`. Notes that Alauda AI 1.5+ performs this scheduling automatically.
- [docs/en/deploy/inference_service/guides/autoscale_settings.mdx](docs/en/deploy/inference_service/guides/autoscale_settings.mdx): Configures Knative Pod Autoscaler (KPA) — the default for KServe `InferenceService` predictors — covering scale-down (per-service `spec.predictor.minReplicas: 0/1`, cluster-wide `enable-scale-to-zero` in the `config-autoscaler` ConfigMap, `scale-to-zero-pod-retention-period`, `scale-to-zero-grace-period`) and scale-up (soft-limit `scaleTarget` with `scaleMetric: concurrency`, hard-limit `containerConcurrency`, `target-utilization-percentage`, and switching the metric to `rps` via `requests-per-second-target-default`). Stresses keeping the `helm.sh/resource-policy: keep` annotation on the global ConfigMap so customizations survive upgrades.
- [docs/en/deploy/inference_service/guides/create_inference_service_cli.mdx](docs/en/deploy/inference_service/guides/create_inference_service_cli.mdx): Walkthrough for authoring an `InferenceService` YAML by hand and applying it with `kubectl`, using a Qwen2.5-0.5B-Instruct + `aml-vllm-0.9.2-cuda-12.6` runtime as the worked example. Shows the required AML annotations and labels (`aml-model-repo`, `aml-model-group`, `aml.cpaas.io/runtime-type: vllm`, `service.subdomain`), a GPU-aware bash command that auto-detects GPUs, handles GGUF model files, and launches `vllm.entrypoints.openai.api_server` with `--tensor-parallel-size`, CUDA-version `nodeAffinity`, NVIDIA fractional-GPU resource keys (`nvidia.com/gpualloc`, `nvidia.com/gpucores`, `nvidia.com/gpumem`), and a curl test against the OpenAI-compatible `/v1/chat/completions` endpoint.
- [docs/en/deploy/inference_service/guides/custom_inference_runtime.mdx](docs/en/deploy/inference_service/guides/custom_inference_runtime.mdx): Guide to adding custom inference runtimes for serving LLMs and other models beyond the built-in vLLM engine. Walks the standard workflow using Xinference to create runtime resources and select them when publishing an inference service, then gives specific examples for MLServer, Triton Inference Server, vLLM-ascend, and MindIE on Ascend NPU, with a comparison of runtime configurations.
- [docs/en/deploy/inference_service/guides/external_access_inference_service.mdx](docs/en/deploy/inference_service/guides/external_access_inference_service.mdx): Steps to expose an in-cluster inference service to outside callers: locate the URL on the service detail page or `status.url`, create a matching Domain (Network > Domains) scoped to the cluster and project, provision a shared Load Balancer (Network > Load Balancers), then add HTTP/HTTPS listeners and a forwarding rule that targets the `knative-ingressgateway` Service in the `istio-system` namespace (using `knative-serving-cert` as the default certificate for HTTPS). Ends with a `curl --resolve` recipe against `/v1/models` for verifying that the load balancer routes inference traffic correctly.
- [docs/en/deploy/inference_service/guides/index.mdx](docs/en/deploy/inference_service/guides/index.mdx): Index page for the Hardware Profile Guides section that renders an Overview component linking to the underlying admin task pages such as the HardwareProfile create/update/delete walkthrough used by Alauda AI inference deployments.
- [docs/en/deploy/inference_service/guides/keda_autoscaling.mdx](docs/en/deploy/inference_service/guides/keda_autoscaling.mdx): Replaces the default KServe HPA with KEDA-based autoscaling driven by vLLM-specific Prometheus metrics (`vllm:num_requests_running`, `vllm:num_requests_waiting`, `vllm:gpu_cache_usage_perc`, latency histograms). Procedure covers granting `kserve-controller-manager` access to KEDA CRDs (`ClusterRole`/`ClusterRoleBinding`), stopping the service with `serving.kserve.io/stop=true`, copying credentials from the `kube-prometheus-alertmanager-basic-auth` secret in `cpaas-system` into the workload namespace, creating a `TriggerAuthentication`, and adding `serving.kserve.io/autoscalerClass: keda` plus an `autoScaling.metrics` block with a PromQL query and per-replica target so KEDA computes `ceil(metricValue / value)` replicas.
- [docs/en/deploy/inference_service/guides/kubeflow_kserve_endpoints.mdx](docs/en/deploy/inference_service/guides/kubeflow_kserve_endpoints.mdx): Guide to deploying and managing inference services from the KServe Endpoints page in the Kubeflow central dashboard. Creates the same `serving.kserve.io/v1beta1` `InferenceService` object as the CLI and the Alauda AI console, so services are visible in all three; covers accessing the Endpoints UI, deploying a model, and verification.
- [docs/en/deploy/inference_service/guides/kueue_scheduling.mdx](docs/en/deploy/inference_service/guides/kueue_scheduling.mdx): Worked example of using Alauda Build of Kueue to gate KServe InferenceService deployments in Alauda AI, including creating ClusterQueue, ResourceFlavor, and LocalQueue resources that cover CPU, memory, pods, ephemeral-storage, and Alauda Build of Hami vGPU resources (nvidia.com/gpualloc, total-gpucores, total-gpumem). Demonstrates labeling the InferenceService with kueue.x-k8s.io/queue-name, observing predictor pods stuck in SchedulingGated when quotas are insufficient, and unblocking them by raising the nominalQuota for nvidia.com/total-gpucores.
- [docs/en/deploy/inference_service/guides/using_modelcar.mdx](docs/en/deploy/inference_service/guides/using_modelcar.mdx): Packages model artifacts as OCI container images (KServe Modelcar) and references them with `storageUri: oci://...` to gain faster cold starts, lower disk churn, and offline distribution via Harbor/Quay. Provides two Containerfile templates (busybox and Red Hat `ubi-micro`), Podman/nerdctl build-and-push commands, advice on splitting large `.safetensors` files across layers to respect Harbor's per-layer size limit, and a sample `InferenceService` that consumes the image with the vLLM runtime; troubleshooting tips cover `kubectl describe`, predictor logs, and `crictl pull` for verifying registry access from a node.
- [docs/en/deploy/inference_service/guides/vllm_expert_parallel.mdx](docs/en/deploy/inference_service/guides/vllm_expert_parallel.mdx): Single-node YAML pattern for turning on vLLM Expert Parallel (EP) for Mixture-of-Experts models by adding `--enable-expert-parallel`, `--tensor-parallel-size 1`, and `--data-parallel-size "${GPU_COUNT}"` to the predictor command, with EP size derived as `TP_SIZE x DP_SIZE`. Explains the layer-level consequences (expert layers sharded across all EP ranks, attention layers replicated when `TP_SIZE=1` or sharded otherwise), shows a full `InferenceService` example for DeepSeek-V3-0324, and explicitly defers performance tuning and multi-node deployments to upstream vLLM documentation.
- [docs/en/deploy/inference_service/guides/vllm_speculative_decoding.mdx](docs/en/deploy/inference_service/guides/vllm_speculative_decoding.mdx): Practitioner's guide to enabling vLLM speculative decoding on an `InferenceService` via the single `--speculative-config` JSON argument, focused on N-gram and EAGLE-3 methods with starting `num_speculative_tokens` recommendations. Documents three ways to deliver the target + draft artifacts (KServe `storageUris` plural for 0.16+, single OCI Modelcar bundle, or pre-staged PVC), how to verify acceptance from the `SpecDecoding metrics` log lines and `/metrics` Prometheus counters, a benchmark template, and a rollback path. Includes internal A30 + Qwen3-8B and Llama-3.1-8B + EAGLE3 lab snapshots showing ~1.84x speedup on code-refactor but break-even or regression on chat, plus troubleshooting for OOM, NotReady due to slow cold start, and silently ignored `min_p`/`logit_bias` parameters.
- [docs/en/deploy/inference_service/index.mdx](docs/en/deploy/inference_service/index.mdx): Top-level Inference Service section landing page rendering the `<Overview />` index for the Introduction, Functions, How-To, and Troubleshooting subsections.
- [docs/en/deploy/inference_service/inference_service.mdx](docs/en/deploy/inference_service/inference_service.mdx): Reference for AML's Inference Service feature, which deploys trained models via KServe `InferenceService` CRD using runtimes such as vLLM, Seldon MLServer, and `llm-d`. Walks through publishing flows (Custom Publish and Template Publish, from Model Repository or PVC), template lifecycle, rolling updates, the visual Inference Experience for text generation, text/image classification, and text-to-image tasks, plus HTTP API/gRPC invocation. Includes a full parameter table for publishing (Hardware Profile vs. Custom resources, GPU allocation, autoscaling, env vars, startup command) and inference-time generation parameters (`max_new_tokens`, `temperature`, `top_k`/`top_p`, beam search, contrastive search, diffusion `num_inference_steps`, `guidance_scale`).
- [docs/en/deploy/inference_service/intro.mdx](docs/en/deploy/inference_service/intro.mdx): One-page introduction framing the Inference Service as Alauda AI's core LLM-serving capability, exposing models over HTTP API and gRPC for building stable, high-performance LLM applications. Carries a warning that the built-in runtime container requires root privileges and should only be used in trusted environments under existing security policies.
- [docs/en/deploy/inference_service/troubleshooting/index.mdx](docs/en/deploy/inference_service/troubleshooting/index.mdx): Section landing page that renders an `<Overview />` index of inference-service troubleshooting topics (currently MLServer timeout diagnostics).
- [docs/en/deploy/inference_service/troubleshooting/infer_timeout.mdx](docs/en/deploy/inference_service/troubleshooting/infer_timeout.mdx): Diagnoses inference-experience `502 Bad Gateway` timeouts on the Seldon MLServer runtime, attributing them to insufficient compute, overly long generated outputs, or MLServer's non-streaming response behavior that blocks until the full inference completes. Recommends upgrading CPU/GPU/memory, capping `max_new_tokens` at request time, and reducing model size/input complexity via quantization, pruning, or input preprocessing to bring response times back under the gateway's timeout window.
- [docs/en/deploy/inference_service/troubleshooting/pod_security_admission_violation.mdx](docs/en/deploy/inference_service/troubleshooting/pod_security_admission_violation.mdx): Troubleshoots inference services stuck out of Running state on Alauda Container Platform when the Deployment exists but no Pod is created and FailedCreate events cite Pod Security Admission baseline violations such as hostIPC=true. Walks through inspecting the runtime for privileged requirements like host namespaces, removing them when possible, or otherwise raising the namespace's Enforce/Audit/Warn Pod Security Standard to Privileged via the Project view's Update Pod Security Admission action.
- [docs/en/deploy/maas/intro.mdx](docs/en/deploy/maas/intro.mdx): Overview of Model as a Service (MaaS), a managed OpenAI-compatible model-serving gateway built on Alauda Build of Envoy Gateway, Envoy AI Gateway, and Authorino. Introduces the MaaS value proposition, request flow, core concepts, roles, model sources, and links to role-specific guides.
- [docs/en/deploy/maas/administrator.mdx](docs/en/deploy/maas/administrator.mdx): Administrator guide for MaaS. Covers publishing platform `LLMInferenceService` models, importing external model services and credentials, publishing external model names, creating subscriptions and per-model token quotas, configuring guardrails, and reviewing usage reports with CSV export.
- [docs/en/deploy/maas/user.mdx](docs/en/deploy/maas/user.mdx): Developer and end-user guide for MaaS. Covers opening My Subscriptions, finding accessible models and endpoints, creating API keys with one-time plaintext display, making OpenAI-compatible curl and Python requests, understanding per-user quotas, viewing personal usage, interpreting 401/403/429 errors, and securing keys.
- [docs/en/deploy/model_compression/compressor_by_workbench.mdx](docs/en/deploy/model_compression/compressor_by_workbench.mdx): Guide to running LLM Compressor model compression workflows on Alauda AI from a workbench. Covers the data-free and calibration-dataset example notebooks, creating a workbench and model repository, uploading models and optional datasets including from S3-compatible object storage, cloning them in JupyterLab, running the compression notebooks, uploading the compressed model, and deploying it for inference.
- [docs/en/deploy/model_compression/index.mdx](docs/en/deploy/model_compression/index.mdx): Top-level landing page for the LLM Compressor section, embedding `<Overview />` to surface the introduction and how-to subsections.
- [docs/en/deploy/model_compression/intro.mdx](docs/en/deploy/model_compression/intro.mdx): Introduces LLM Compressor as the vLLM project's open-source model-compression library that supports quantization (int8, W4A16 weight-only, W8A8 weight+activation), sparsity, and file-size compression with native Hugging Face and vLLM integration; lists supported algorithms AWQ, GPTQ, FP8 dynamic per-token, SparseGPT, and SmoothQuant, and explains how computed scales/zero-points (per-tensor, channel, group, or token) produce compressed models suitable for resource-limited deployment.
- [docs/en/deploy/model_management/index.mdx](docs/en/deploy/model_management/index.mdx): Section landing page for the Dify overview, embedding the Overview component to list the Introduction and Main Features sub-pages.
- [docs/en/deploy/model_management/intro.mdx](docs/en/deploy/model_management/intro.mdx): Brief introduction positioning the Model Repository as a Git LFS-based versioned storage hub for LLMs on the Alauda AI platform, covering inference deployment, fine-tuning, and evaluation phases with version, sharing, and lifecycle management capabilities.
- [docs/en/deploy/model_management/model_repository.mdx](docs/en/deploy/model_management/model_repository.mdx): Reference for the Git-backed Model Repository: creating and deleting repos with name/description/visibility metadata, pushing large files via Git LFS using `.gitattributes` patterns like `*.bin filter=lfs`, branching/tagging for parallel versions, auto-syncing README.md metadata, and cross-tenant Shared/`public` namespace publishing. Also covers integrations including one-click inference service launch from tagged versions and pulling models into AML Notebooks via `git clone`.
- [docs/en/deploy/model_management/model_storage.mdx](docs/en/deploy/model_management/model_storage.mdx): Reference for the model storage types Alauda AI supports before a model can be deployed: S3-compatible object storage downloaded by a storage initializer init container, Persistent Volume Claims mounted before the main container starts, and OCI containers (KServe modelcars) that achieve fast loading through container runtime layer caching. Covers authentication configuration and inference service deployment for each.
- [docs/en/deploy/model_management/share_models.mdx](docs/en/deploy/model_management/share_models.mdx): Manual workflow for sharing an LLM that lacks a UI publishing flow: add model card metadata (`pipeline_tag`, `library_name`) either through the File Management Edit Metadata dialog or by writing a YAML frontmatter block into the model's README.md, then change the backing GitLab project's visibility from Private to Public. When using README-based metadata, GitLab admin must also add project topics like `text-classification` and `aml_framework_transformers` so the platform recognizes the model as inference-ready; metadata cannot be edited once shared.
- [docs/en/deploy/model_management/upload_models_using_notebook.mdx](docs/en/deploy/model_management/upload_models_using_notebook.mdx): End-to-end procedure for uploading an LLM into the Alauda AI model repository from a Workbench/Notebook, including downloading from huggingface.co, hf-mirror.com, or modelscope.cn with `huggingface-cli`, initializing a Git repo, writing a `.gitattributes` file that LFS-tracks formats like `*.safetensors`, `*.bin`, `*.pt`, and `*.ckpt`, and pushing with `git -c lfs.activitytimeout=36000 push -u origin main`. Also covers editing Task Type/Framework metadata (e.g., text-generation + Transformers) and using `git lfs migrate import --above 100MB` to retroactively LFS-track large files.

## develop

- [docs/en/develop/agentic_mlops/coding-agents-with-inference-service.mdx](docs/en/develop/agentic_mlops/coding-agents-with-inference-service.mdx): Guide to pointing terminal coding agents such as opencode, Codex CLI, and Claude Code at a model served on Alauda AI so source code, prompts, and infrastructure configuration never leave the cluster. Covers deploying and smoke-testing the endpoint, enabling tool calling on the runtime, configuring reasoning models and reasoning effort, and connecting each agent.
- [docs/en/develop/agentic_mlops/index.mdx](docs/en/develop/agentic_mlops/index.mdx): Section index for Agentic MLOps, linking the guides that connect coding agents to on-premise inference services and use them to drive day-to-day MLOps on the platform.
- [docs/en/develop/agentic_mlops/mlops-with-coding-agents.mdx](docs/en/develop/agentic_mlops/mlops-with-coding-agents.mdx): Guide to using a coding agent wired to a self-hosted model for day-to-day MLOps on Alauda AI. Covers setting up the agent's working environment, managing InferenceServices and LLMInferenceServices, managing gateway authentication and rate limits, tuning service performance against service-level objectives with reproducible benchmarks, and planning fine-tuning with a reusable template.
- [docs/en/develop/components/data_science_pipelines/index.mdx](docs/en/develop/components/data_science_pipelines/index.mdx): Component landing page for the Data Science Pipelines Operator, linking its introduction and installation guide.
- [docs/en/develop/components/data_science_pipelines/install.mdx](docs/en/develop/components/data_science_pipelines/install.mdx): Installs the Data Science Pipelines Operator as an OLM operator from the platform OperatorHub. Lists prerequisites including ACP v4.0 or later, amd64 or arm64 clusters, OLM, mutual exclusion with the shared `kfp-operator` install, and the Argo Workflow CRDs; covers uploading and installing the operator, the optional external endpoint provider, and verifying the CSV, controller pod, and CRDs.
- [docs/en/develop/components/data_science_pipelines/intro.mdx](docs/en/develop/components/data_science_pipelines/intro.mdx): Introduction to the Data Science Pipelines Operator, a Kubernetes-native operator built on the OpenDataHub project that deploys and manages Kubeflow Pipelines v2 stacks. Each data-science project runs its own isolated pipeline runtime declared through a `DataSciencePipelinesApplication` custom resource. Covers what it deploys, key concepts, mutual exclusion with the shared Kubeflow Pipelines install, and when to use it.
- [docs/en/develop/components/feast/index.mdx](docs/en/develop/components/feast/index.mdx): Component landing page for Alauda Build of Feast, linking its introduction, installation guide, and quickstart.
- [docs/en/develop/components/feast/install.mdx](docs/en/develop/components/feast/install.mdx): Installs Alauda Build of Feast by uploading the Feast Operator package and setting the `feast` component to `Managed` in the `default` `AmlCluster`; the operator registers the `featurestores.feast.dev` CRD. After the component is reconciled, a `FeatureStore` custom resource deploys Feast services per the Quickstart.
- [docs/en/develop/components/feast/intro.mdx](docs/en/develop/components/feast/intro.mdx): Introduces Feast as an open-source feature store deployed on Kubernetes through the Feast Operator, where a FeatureStore CR manages the offline store, online store, registry, UI, and a generated client feature_store.yaml ConfigMap. Defines core concepts (Project via spec.feastProject, Entity, Data Source, Feature View, Feature Service, Registry, Materialization, Push Source, Permission), outlines the typical workflow from feast apply through materialization to client reads, and describes Operator responsibilities such as PVC provisioning, Git-based feature repo init, and Kubernetes Role creation for RoleBasedPolicy authorization.
- [docs/en/develop/components/feast/quickstart.mdx](docs/en/develop/components/feast/quickstart.mdx): End-to-end guide for configuring and deploying a FeatureStore custom resource after the Feast Operator is installed, covering the main spec fields (feastProject, feastProjectDir, services.offlineStore/onlineStore/registry/ui, authz, replicas) and runnable YAML for persistence patterns: PVC-backed DuckDB plus local file registry, Redis online store with SQL registry on PostgreSQL 16, PostgreSQL-only online plus registry, and registry files in S3/GCS. Also documents Secret layout per backend type (redis, postgres, sql keys), feature repository initialization via feast init/init template/Git, deploying with kubectl apply and watching status.phase, the feast-<name>-online/offline/registry/ui Service names and clientConfigMap, using the Feast CLI/Python SDK with feast apply, and Kubernetes-based authorization combining feast-reader/feast-writer roles, Feast Permission objects with RoleBasedPolicy, RoleBindings, ServiceAccount tokens, and SDK token configuration via authz_config.user_token or LOCAL_K8S_TOKEN.
- [docs/en/develop/components/index.mdx](docs/en/develop/components/index.mdx): Section index for the components the Develop chapter relies on, gathering the introduction and installation guides for Kubeflow, Data Science Pipelines, MLflow, KubeRay, Spark Operator, Label Studio, and Feast.
- [docs/en/develop/components/kubeflow/faq.mdx](docs/en/develop/components/kubeflow/faq.mdx): Troubleshooting FAQ for Kubeflow on Alauda AI covering four scenarios: relabeling namespaces from PSA `restricted` to `baseline` to unblock notebooks and pipeline runs; overriding the `oidcAuthURL` in the `kfbase` `ModuleInfo` resource and restarting `oauth2-proxy` when login must use an alternative platform address; creating a `kfp-launcher` ConfigMap to point Kubeflow Pipelines runs at external S3/MinIO instead of the in-cluster `minio-service.kubeflow:9000`; adding custom GPU vendors (e.g. `your-custom.com/gpu`, Ascend) to `jupyter-web-app-config`; and fixing `CrashLoopBackOff`/probe timeouts on kube-ovn CNI by adding an `allow-kubelet-probes` NetworkPolicy that permits ingress from the kube-ovn join subnet CIDR (typically `100.64.0.0/16`).
- [docs/en/develop/components/kubeflow/index.mdx](docs/en/develop/components/kubeflow/index.mdx): Component landing page for Alauda support for Kubeflow, linking its introduction, installation guide, upgrade guide, and FAQ.
- [docs/en/develop/components/kubeflow/install.mdx](docs/en/develop/components/kubeflow/install.mdx): Deployment guide for the Kubeflow OLM operators on Alauda AI 2.3+: `kfbase-operator`, `kfp-operator`, `kubeflow-trainer-operator` (replacing the deprecated `kftraining` plugin), and `model-registry-operator`. Covers prerequisites including ASM v2, LWS, and Alauda Build of KServe, configuring Dex redirection via Platform Access URLs, wiring oauth2-proxy through ASM extensionProviders (envoyExtAuthzHttp), downloading and uploading the operator bundles, creating Kubeflow Profile resources to bind users to namespaces, and installing the Model Registry operator with MySQL storage settings.
- [docs/en/develop/components/kubeflow/intro.mdx](docs/en/develop/components/kubeflow/intro.mdx): Brief introduction positioning Alauda's Kubeflow integration as a Kubernetes-native ML platform combining Kubeflow Pipelines for workflow orchestration, Kubeflow Training for jobs, and Model Registry for versioning, and notes that namespaces must have Pod Security Admission set to privileged for Kubeflow components to function.
- [docs/en/develop/components/kubeflow/upgrade.mdx](docs/en/develop/components/kubeflow/upgrade.mdx): Captures manual post-upgrade actions for the kfbase plugin, including the switch from NodePort to gateway-based dashboard access for upgrades from v1.10.13 or earlier (requiring DNS or hosts updates pointing kubeflowDomain to the kubeflow-external-gateway IP), instructions for re-enabling NodePort by editing the kubeflow-istio-ingressgateway service, and the v1.10.10 requirement to set a default StorageClass for the pgStorageClass parameter when upgrading from v1.10.9 or earlier.
- [docs/en/develop/components/kuberay/index.mdx](docs/en/develop/components/kuberay/index.mdx): Component landing page for Alauda Build of KubeRay Operator, linking its introduction and installation guide.
- [docs/en/develop/components/kuberay/install.mdx](docs/en/develop/components/kuberay/install.mdx): Installation guide for the Alauda Build of KubeRay Operator cluster plugin on ACP v4.0 or later, covering downloading the package from the Customer Portal and uploading it following the shared Upload Packages guide, deploying it from Administrator > Marketplace > Cluster Plugin to the target cluster, and verifying with kubectl get pods -n cpaas-system | grep kuberay-operator.
- [docs/en/develop/components/kuberay/intro.mdx](docs/en/develop/components/kuberay/intro.mdx): Introduces the Alauda Build of KubeRay Operator as a Kubernetes-native operator built on the open-source KubeRay project for running Ray on Kubernetes. Describes the three core CRDs (RayCluster for lifecycle and autoscaling, RayJob for auto-provisioned job execution with cleanup, RayService for zero-downtime Ray Serve deployments), key features like heterogeneous compute, fault tolerance, and ecosystem integration with Kueue/Volcano/Prometheus, and use cases spanning distributed training, batch inference, hyperparameter tuning with Ray Tune, and LLM serving.
- [docs/en/develop/components/label_studio/features.mdx](docs/en/develop/components/label_studio/features.mdx): Lists Label Studio's main feature areas: multi-user annotation with user management, collaborative labeling, task assignment, and quality control; multi-type data support spanning image (classification, object detection, semantic segmentation), text, audio, video, time-series, and multi-modal data; XML-based annotation configuration with template library and built-in tools; JSON/CSV data import/export with batch operations; and machine learning integration via ML backend, pre-annotation, REST API, and Python SDK.
- [docs/en/develop/components/label_studio/index.mdx](docs/en/develop/components/label_studio/index.mdx): Component landing page for Label Studio, linking its introduction, installation guide, feature reference, and quickstart.
- [docs/en/develop/components/label_studio/install.mdx](docs/en/develop/components/label_studio/install.mdx): Step-by-step guide for deploying Label Studio to a Kubernetes cluster: download and upload the `label-studio.ALL.xxxx.tgz` package, prepare a CSI/`PersistentVolume`-backed StorageClass and a PostgreSQL 13+ cluster (and optional standalone-mode Redis from Data Services), then install the `3rdparty/chart-label-studio` Catalog application. Covers custom values for persistence sizing, `pgConfig`/`redisConfig` with SSL secrets, Service type and Ingress with `LABEL_STUDIO_HOST`, OAuth2 Proxy with ACP Dex as OIDC provider (via an `OAuth2Client` CR), and disabling open registration with `LABEL_STUDIO_DISABLE_SIGNUP_WITHOUT_LINK`.
- [docs/en/develop/components/label_studio/intro.mdx](docs/en/develop/components/label_studio/intro.mdx): Introduces Label Studio as an open-source multi-type data labeling tool with a Django/Python REST backend, React frontend, PostgreSQL 13+ storage, and optional Redis cache. Explains the core concepts (Project, Labeling Interface, Data Manager, Annotations, Machine Learning Integration), supported data types (image, audio, text, time series, video), cloud storage import from AWS S3 and Google Cloud Storage, and capabilities like pre-annotation, online/active learning, and model comparison.
- [docs/en/develop/components/label_studio/quickstart.mdx](docs/en/develop/components/label_studio/quickstart.mdx): Walks users through integrating Label Studio with S3-compatible object storage (Amazon S3, Ceph RGW) for data import and annotation export. Covers configuring Source/Target Cloud Storage under `Settings > Cloud Storage`, fields like Bucket Name, S3 Endpoint, Access Key ID/Secret, Bucket Prefix, File Filter Regex, pre-signed URL expiration, and SSE KMS Key ID; syncing buckets, exporting JSON annotations, and using the Label Studio SDK converter to transform them into COCO, Pascal VOC, YOLO, or CSV for downstream model training pipelines.
- [docs/en/develop/components/mlflow/index.mdx](docs/en/develop/components/mlflow/index.mdx): Component landing page for MLflow, linking its introduction and installation guide.
- [docs/en/develop/components/mlflow/install.mdx](docs/en/develop/components/mlflow/install.mdx): Installs MLflow in Alauda AI 2.8.x by uploading the MLflow Operator package and setting `spec.components.mlflow.managementState` to `Managed` in the `default` `AmlCluster`; enabling the component installs the MLflow Operator only. Then creates one `MLflow` custom resource that reconciles the tracking-server stack. Lists prerequisites including Alauda AI, OLM, and a PostgreSQL 12 or later database; covers both the kubectl and Alauda Console UI methods, verification of the operand, Deployment, and pod, and high availability and storage options.
- [docs/en/develop/components/mlflow/intro.mdx](docs/en/develop/components/mlflow/intro.mdx): Introduction to the MLflow Operator, which deploys and manages a multi-tenant MLflow Tracking Server integrated with platform single sign-on and Kubernetes RBAC so each team sees only its own runs, experiments, and registered models. Covers what it deploys, key concepts, the multi-tenancy model, and when to use it.
- [docs/en/develop/components/spark_operator/index.mdx](docs/en/develop/components/spark_operator/index.mdx): Component landing page for Alauda Build of Spark Operator, linking its introduction and installation guide.
- [docs/en/develop/components/spark_operator/install.mdx](docs/en/develop/components/spark_operator/install.mdx): Installs Alauda Build of Spark Operator as an OLM operator from the platform OperatorHub. Lists prerequisites including ACP v4.0 or later, amd64 or arm64 clusters, and OLM; covers uploading and installing the operator and verifying that the CSV reports Succeeded, the controller and webhook pods are Running, and the CRDs are registered.
- [docs/en/develop/components/spark_operator/intro.mdx](docs/en/develop/components/spark_operator/intro.mdx): Introduction to Alauda Build of Spark Operator, a Kubernetes-native operator built on the Kubeflow Spark Operator that submits, schedules, monitors, and cleans up Apache Spark workloads declaratively through `sparkoperator.k8s.io` CRDs, without a spark-submit client or a standalone Spark cluster. Covers key features and use cases.
- [docs/en/develop/connections/index.mdx](docs/en/develop/connections/index.mdx): Top-level landing page for the Connections section in the Alauda AI documentation, rendered via the Overview component to surface its child pages.
- [docs/en/develop/connections/intro.mdx](docs/en/develop/connections/intro.mdx): Introduces Alauda AI Connections, project-scoped reusable access settings for external model sources and data services that are stored as Kubernetes Secrets in the project namespace, while Connection Types (ConfigMaps in kube-public) define the form fields users see. Covers the three supported kinds (URI, OCI-compliant registry, S3-compatible object storage) and how each is consumed by the model deployment form (URI passthrough, image pull secret plus repository:tag path, or bucket plus object path).
- [docs/en/develop/connections/using_connections.mdx](docs/en/develop/connections/using_connections.mdx): Task guide for the Connections feature: how to create, edit, and delete project-level connections (URI, OCI registry, S3-compatible storage) under a project's **Connections** tab, manage cluster-scoped connection types in the `kube-public` namespace, and reference a connection during InferenceService creation via the `aml-model-source-connection` annotation. Includes built-in field-set details (`AWS_ACCESS_KEY_ID`, `OCI_HOST`, `.dockerconfigjson` upload), runtime behavior (imagePullSecret for OCI, ServiceAccount for S3), and YAML examples for connection secrets and `ct-*` ConfigMap templates.
- [docs/en/develop/distributed_workloads/codeflare-sdk-tutorial.mdx](docs/en/develop/distributed_workloads/codeflare-sdk-tutorial.mdx): End-to-end tutorial for using the CodeFlare SDK inside a Standard Data Science workbench in Alauda AI to spin up a RayCluster via ClusterConfiguration, verify it with cluster.status(), submit a RayJob with parameters such as job_name, cluster_name, and entrypoint, monitor it with rayjob.status(), and tear down with cluster.down(). References a downloadable demo Jupyter notebook and reminds users to update the image parameter for their hardware and registry.
- [docs/en/develop/distributed_workloads/index.mdx](docs/en/develop/distributed_workloads/index.mdx): Section index for distributed workloads, linking the guides that run distributed Ray and Spark workloads from a workbench or from the command line.
- [docs/en/develop/distributed_workloads/run_spark_application.mdx](docs/en/develop/distributed_workloads/run_spark_application.mdx): Guide to submitting an Apache Spark job with a `SparkApplication` custom resource using the built-in Spark Pi example. Covers creating a namespace and Spark RBAC, submitting the application, monitoring high-level status and driver and executor pods, verifying the result, cleanup, scheduling a recurring job, and common `SparkApplication` fields.
- [docs/en/develop/experiment_tracking/agent-tracing.mdx](docs/en/develop/experiment_tracking/agent-tracing.mdx): Guide to recording AI agent traces with MLflow Tracing, capturing model requests and responses, chain and tool calls, retrieval steps, latency, token usage, and errors under the same workspace authentication and Kubernetes RBAC as runs and models. Covers tracing LangChain agents and OGX applications, exporting OGX OpenTelemetry traces, viewing traces, and troubleshooting.
- [docs/en/develop/experiment_tracking/index.mdx](docs/en/develop/experiment_tracking/index.mdx): Section index for experiment tracking, linking the MLflow Python SDK authentication guide, Kubeflow Pipelines integration, and AI agent tracing.
- [docs/en/develop/experiment_tracking/mlflow-python-sdk.mdx](docs/en/develop/experiment_tracking/mlflow-python-sdk.mdx): Guide to driving the stock MLflow Python SDK through the platform OAuth proxy browser-free, using the OAuth2 authorization code flow with PKCE scripted against the platform login. Covers how authentication works, verifying the token method as an administrator, obtaining a token from the command line with a Python helper or curl and openssl, connecting the SDK, selecting a workspace, registering models, and a session-cookie alternative.
- [docs/en/develop/experiment_tracking/pipelines-mlflow-integration.mdx](docs/en/develop/experiment_tracking/pipelines-mlflow-integration.mdx): Guide to logging parameters, metrics, and models from Kubeflow Pipelines components to MLflow on Alauda AI with the MLflow Python client, authenticating each component with a user identity token so runs are recorded under that user. Covers a complete training pipeline example, uploading and running it from the KFP UI or SDK, using MLflow in Trainer v2 pipelines, and best practices.
- [docs/en/develop/index.mdx](docs/en/develop/index.mdx): Chapter landing page for data scientists and application developers, covering workbenches and Kubeflow notebooks, connections, pipeline orchestration, the model registry, experiment tracking, and distributed computing, with the chapter's components listed under Components.
- [docs/en/develop/kubeflow_notebooks/index.mdx](docs/en/develop/kubeflow_notebooks/index.mdx): Section index for Kubeflow Notebooks, covering notebook servers, volumes, and TensorBoards from the Kubeflow central dashboard. Positions Kubeflow Notebooks as an alternative to Workbench for the lightweight upstream setup, and notes that these pages require the Kubeflow operators.
- [docs/en/develop/kubeflow_notebooks/notebooks.mdx](docs/en/develop/kubeflow_notebooks/notebooks.mdx): Guide to using **Kubeflow Notebooks** (with a note recommending Alauda AI Workbench instead for richer integration): create per-namespace JupyterLab, VS Code, or RStudio servers with custom Docker images, CPU/RAM/GPU requests, workspace and data PVCs, and PodDefault-injected configurations. Covers persisting Python virtual environments under `/home/jovyan/venv` (registering them as Jupyter kernels via `ipykernel`), applying `PodDefault` resources in the `kubeflow.org/v1alpha1` API to inject credentials such as a `gcp-secret` volume mount, and reading data from object storage with `s3fs` using `AWS_S3_ENDPOINT`/`AWS_ACCESS_KEY_ID` environment variables.
- [docs/en/develop/kubeflow_notebooks/tensorboards.mdx](docs/en/develop/kubeflow_notebooks/tensorboards.mdx): Walks through spawning **Kubeflow TensorBoards** that point at training logs on PVCs or object storage URIs like `s3://my-bucket/logs/experiment-1`. Includes a PyTorch `SummaryWriter` example that logs an image grid and the ResNet50 model graph to `./runs/`, instructions for configuring the TensorBoard instance (Name, PVC source with Mount Path, or Object Store Link plus a credentials PodDefault), and usage scenarios such as comparing multiple runs via a parent log directory and inspecting the **Scalars** and **Graphs** tabs. Notes that deleting an instance reclaims CPU/memory but never touches the underlying logs.
- [docs/en/develop/kubeflow_notebooks/volumes.mdx](docs/en/develop/kubeflow_notebooks/volumes.mdx): Guide to managing Kubeflow volumes, which are Kubernetes Persistent Volume Claims that provide storage for data, workspaces, and models independent of the notebook server lifecycle. Covers creating a volume with a name, storage class, and size from the Kubeflow central dashboard, managing existing volumes, and attaching a volume to a notebook.
- [docs/en/develop/model_registry.mdx](docs/en/develop/model_registry.mdx): How-to for the **Kubeflow Model Registry**: register models either via the dashboard UI (model name, S3 artifact URI, version, tags) or programmatically with the `model-registry` Python client (`pip install model-registry==0.3.5 kserve==0.13`) calling `register_model()` with `model_format_name`, `version`, and `metadata` against `http://model-registry-service.<namespace>.svc:8080`. Then shows deploying a registered model as a KServe `V1beta1InferenceService` carrying the `modelregistry/registered-model-id` and `modelregistry/model-version-id` labels so the controller pulls the model from the registry-stored S3 URI.
- [docs/en/develop/pipelines/create_dspa.mdx](docs/en/develop/pipelines/create_dspa.mdx): Guide to creating a `DataSciencePipelinesApplication`, an isolated Kubeflow Pipelines v2 stack in a namespace, and then using it from the kfp Python SDK. Covers choosing a database and object storage with their credential secrets, creating the DSPA with either kubectl or the Alauda Console UI, verifying that it reports Ready and its component pods are Running, and compiling and uploading a pipeline.
- [docs/en/develop/pipelines/index.mdx](docs/en/develop/pipelines/index.mdx): Section index for pipelines, linking guides that create a pipeline server, build and run Kubeflow Pipelines, trigger them from Alauda DevOps, and configure and run Elyra notebook pipelines in an air-gapped Workbench.
- [docs/en/develop/pipelines/kfp-execution-and-storage.mdx](docs/en/develop/pipelines/kfp-execution-and-storage.mdx): Reference for Kubeflow Pipelines 2.16.1 execution and storage behavior as installed by Alauda kfp-operator v26.3.1 or the Data Science Pipelines Operator v2.15.2. Covers task caching, `dsl.ParallelFor`, typed KFP artifacts, and S3-compatible artifact persistence on amd64 and arm64, with a verification pipeline, arm64 deployment notes for both operators, and platform compatibility and database backends.
- [docs/en/develop/pipelines/kubeflow_pipelines.mdx](docs/en/develop/pipelines/kubeflow_pipelines.mdx): Walks through using **Kubeflow Pipelines** (KFP): install the `kfp` Python SDK, configure a `kfp-launcher` ConfigMap so runs use your external Object Storage (`defaultPipelineRoot`, `endpoint`, `forcePathStyle`, secret-based credentials) instead of the Kubeflow 1.11+ default SeaweedFS, and compile a simple `@dsl.pipeline`/`@dsl.component` (`say_hello`) with `compiler.Compiler().compile(...)`, then submit it via `Client.create_run_from_pipeline_package`. Also documents the dashboard workflow for uploading a `pipeline.yaml`, creating one-off or scheduled (Periodic/Cron) Recurring Runs, grouping runs into experiments, and inspecting per-step graph nodes, logs, inputs/outputs, and visualizations.
- [docs/en/develop/pipelines/reusable-pipeline-components.mdx](docs/en/develop/pipelines/reusable-pipeline-components.mdx): Guide to the reusable pipeline components Alauda AI packages for Kubeflow Pipelines 2.16.1, including the beta AutoGluon tabular and time-series training pipelines. Covers creating the input-storage Secret, running each managed pipeline, composing a custom pipeline from the mirrored component library in a Python virtual environment, and troubleshooting.
- [docs/en/develop/pipelines/tekton.mdx](docs/en/develop/pipelines/tekton.mdx): Demonstrates integrating Alauda Build of Kueue scheduling with Alauda DevOps Pipelines (Tekton) and Alauda Build of HAMi vGPU resources by creating a `ClusterQueue`/`ResourceFlavor`/`LocalQueue` that covers `cpu`, `memory`, `pods`, `nvidia.com/gpualloc`, `nvidia.com/total-gpucores`, and `nvidia.com/total-gpumem`, then submitting a Tekton `Pipeline` and `PipelineRun` labelled with `kueue.x-k8s.io/queue-name`. Shows how a quota-exceeding pod stays in `SchedulingGated` state until the GPU core quota is raised, after which it transitions to `Running`.
- [docs/en/develop/workbench/create_workbench.mdx](docs/en/develop/workbench/create_workbench.mdx): Walks through creating a Workbench instance from the Alauda AI console (including PVC prerequisites, the Connect button, and offline `pip install` of uploaded `.whl` files in JupyterLab), then catalogs the built-in and Docker Hub `WorkspaceKind` images for CPU, NVIDIA CUDA, and Ascend CANN (Minimal Python, Standard Data Science, code-server, PyTorch/TensorFlow CUDA, PyTorch LLM Compressor, PyTorch/MindSpore/ModelSlim CANN). Documents the `sync-from-dockerhub.sh` relay tool (nerdctl pull, tar export, skopeo push) with `TARGET_REGISTRY`/`DOCKERHUB_USER` environment variables, JSON-patching `WorkspaceKind.spec.podTemplate.options.imageConfig` to register synced images, and patching `supplementalGroups` so `huawei.com/Ascend910B4` vNPU devices avoid the `dcmi module initialize failed` error.
- [docs/en/develop/workbench/index.mdx](docs/en/develop/workbench/index.mdx): Top-level Workbench section landing page that renders the auto-generated `<Overview />` table of contents for the Workbench documentation tree.
- [docs/en/develop/workbench/intro.mdx](docs/en/develop/workbench/intro.mdx): Concise product introduction explaining that Workbench manages cloud-IDE instances (JupyterLab, code-server, future RStudio) backed by Kubeflow Notebook 2.0, lets developers attach GPUs and reach in-cluster services for ML jobs/pipelines, and uses Kubeflow RBAC for sharing; also defines the cluster-admin-managed `WorkspaceKind` CRD (currently `podTemplate`-kind, extensible to others like `helmTemplate`) as the template that drives each Workspace.

## evaluate_safety

- [docs/en/evaluate_safety/ai_guardrails.mdx](docs/en/evaluate_safety/ai_guardrails.mdx): Deploys the TrustyAI GuardrailsOrchestrator CRD (based on FMS-Guardrails) in AutoConfig mode with the built-in regex detector and gateway enabled, including supported algorithms (email, US SSN, credit-card, IPv4/IPv6, US phone, UK postcode, custom regex). Documents the gateway, built-in-detector, orchestrator, and health ports under auth-enabled/disabled modes, ServiceAccount Bearer-token setup, and curl examples for /api/v1/text/contents, /api/v2/chat/completions-detection, and /all/v1/chat/completions including PII detection response shapes.
- [docs/en/evaluate_safety/components/index.mdx](docs/en/evaluate_safety/components/index.mdx): Section index for the components the Evaluate and Safety chapter relies on, gathering the introduction and installation guides for TrustyAI.
- [docs/en/evaluate_safety/components/trustyai/deploy_tas.mdx](docs/en/evaluate_safety/components/trustyai/deploy_tas.mdx): Guide to deploying the TrustyAI Service, which runs alongside KServe models and collects the inference data that drift detection and bias metrics are computed from. Covers creating the `TrustyAIService` resource in either DATABASE mode with a MySQL 8.x database or PVC mode with a working default StorageClass, and verifying deployment readiness.
- [docs/en/evaluate_safety/components/trustyai/index.mdx](docs/en/evaluate_safety/components/trustyai/index.mdx): Component landing page for Alauda Build of TrustyAI, linking its introduction, installation guide, and TrustyAI Service deployment guide.
- [docs/en/evaluate_safety/components/trustyai/install.mdx](docs/en/evaluate_safety/components/trustyai/install.mdx): Installs Alauda Build of TrustyAI by uploading the TrustyAI Operator package and setting `spec.components.trustyAI.managementState` to `Managed` in the `default` `AmlCluster`; the operator registers the TrustyAI CRDs. Lists the CRDs (TrustyAIService, LMEvalJob, GuardrailsOrchestrator, NemoGuardrails) made available for inference monitoring, LLM evaluation, and AI guardrails orchestration.
- [docs/en/evaluate_safety/components/trustyai/intro.mdx](docs/en/evaluate_safety/components/trustyai/intro.mdx): Introduction to Alauda Build of TrustyAI, based on the TrustyAI Kubernetes operator, which simplifies deploying model explainability, fairness monitoring, LLM evaluation, and AI guardrails. Describes the TrustyAI Service that collects KServe inference data, LM-Eval for model evaluation, and the guardrails components, and links their respective guides.
- [docs/en/evaluate_safety/index.mdx](docs/en/evaluate_safety/index.mdx): Chapter landing page for evaluation and safety, covering model and RAG evaluation and guardrails for LLM inputs and outputs, with the chapter's components listed under Components. Notes that bias and drift monitoring with TrustyAI is covered in the Monitor chapter.
- [docs/en/evaluate_safety/lm_eval.mdx](docs/en/evaluate_safety/lm_eval.mdx): Runs LLM evaluations against a KServe InferenceService using the LMEvalJob CRD with the lm-evaluation-harness backend and Unitxt. Shows a full example targeting the arc_easy task with local-completions or local-chat-completions model types, modelArgs (base_url, tokenizer, num_concurrent), allowOnline/allowCodeExecution flags, pvcManaged outputs, and status.state lifecycle (New, Scheduled, Running, Complete). Also covers offline mode with offline.storage.pvcName plus HF_HOME, HF_DATASETS_CACHE, and TRANSFORMERS_CACHE env wiring for air-gapped runs.
- [docs/en/evaluate_safety/nemo_guardrails.mdx](docs/en/evaluate_safety/nemo_guardrails.mdx): Deploys NVIDIA NeMo Guardrails as a service via the NemoGuardrails CRD to front an OpenAI-compatible model with input/output rails. Provides a complete ConfigMap example with config.yaml (sensitive_data_detection for EMAIL_ADDRESS), rails.co Colang flows (check message length, check forbidden words, bot inform blocks), and actions.py Python @action handlers. Covers TLS/CA secret handling via trustyai.opendatahub.io/ca-secret-name, SSL_CERT_FILE, ServiceAccount token auth, and curl examples that hit /v1/chat/completions to demonstrate length, forbidden-word, and PII blocking.
- [docs/en/evaluate_safety/ragas.mdx](docs/en/evaluate_safety/ragas.mdx): Guide to evaluating Retrieval-Augmented Generation systems with Ragas, which scores retriever and generator behavior separately so failures can be attributed to retrieval or to generation drift. Covers what to record for each evaluation example, core and optional RAG metrics, choosing a minimal metric set, calling the Ragas SDK, a runnable notebook, troubleshooting, and interpreting results.

## installation

- [docs/en/installation/ai-cluster.mdx](docs/en/installation/ai-cluster.mdx): Installs Alauda AI on a target cluster. Covers downloading the Alauda AI operator package (and optional dependency packages) and uploading them with `violet`, installing the Alauda AI Operator from OperatorHub, configuring the cluster-scoped `default` `AmlCluster` instance, enabling the managed components through the per-component `managementState` settings (`Managed`, `Unmanaged`, `SharedManaged`, `Removed`), and importing models into a registry.
- [docs/en/installation/ai-essentials.mdx](docs/en/installation/ai-essentials.mdx): Configures the Alauda AI entry point and branding for the single-cluster deployment model (Alauda AI 2.6+). Runs the `global-install.sh` script against the global cluster to create the OAuth2Client, the OIDC credential Secret, and the ProductEntry that registers Alauda AI in the console left navigation. Also covers multi-install display-name overrides via ProductEntry patching and in-product branding through the `ui-config` ConfigMap in the `kubeflow` namespace, and warns that the legacy Alauda AI Essentials cluster plugin must be uninstalled after upgrading from an earlier version.
- [docs/en/installation/index.mdx](docs/en/installation/index.mdx): Chapter landing page for installation, listing the hardware requirements of at least two nodes with 16 cores and 32 GB memory plus GPU and disk capacity for serving, the software requirements, and the ordered installation steps for the platform.
- [docs/en/installation/package-upload.mdx](docs/en/installation/package-upload.mdx): The shared procedure for downloading Alauda AI component packages from the Customer Portal and uploading them to the target cluster with the `violet` packaging tool. Covers downloading the matching package (OLM operator bundle or Cluster Plugin), downloading and making `violet` executable, the `violet push` command with `--platform-address`/`--platform-username`/`--platform-password`/`--clusters`, uploading to an external registry with `--dst-repo`, uploading multiple packages with a shell script, and installing the uploaded component from OperatorHub or Cluster Plugins.
- [docs/en/installation/pre-configuration.mdx](docs/en/installation/pre-configuration.mdx): Pre-install configuration for Alauda AI: provisions the (deprecated but still supported) GitLab service used by Model Management — version 15+ with HTTPS, Git LFS enabled, self-hosted, and access-token expiration disabled — and walks through generating an admin impersonation token with full `api` scope, then creating the `aml-gitlab-admin-token` secret under the `cpaas-system` namespace. Also covers preparing an Alauda Build of Harbor registry for Model Catalog (HTTPS production mode with anonymous pull) and configuring `spec.helmValues.gitlab.webservice.ingress.proxyBodySize: "0"` and `proxyReadTimeout: "3600"` plus related nginx ingress annotations on `GitLabOfficial` CR to fix HTTP 413 errors when pushing large LFS objects to GitLab 18.5+.
- [docs/en/installation/tools.mdx](docs/en/installation/tools.mdx): Explains the AML 1.3+ auto-discovery mechanism for the left-navigation **Tools** menu, which scans all `ConfigMap` resources labeled `aml.cpaas.io/centralMenuItem: "true"` and merges them into `centraldashboard-config` (manual edits are reverted on upgrade). Shows an MLFlow example ConfigMap defining `link`, `parentUid: advanced`, `title`/`titleI18N`, `tags`, and `order` fields, and describes the `aml-feature-tags` whitelist with the `aml.cpaas.io/centralTagsFilter` annotation that hides any menu item whose tags fall outside the filter.
- [docs/en/installation/workbench.mdx](docs/en/installation/workbench.mdx): Installs Workbench in Alauda AI 2.8.x, where it is managed by Alauda AI. Covers prerequisites including Alauda AI and OLM, setting `spec.components.workbench.managementState` to `Managed` in the `default` `AmlCluster` (the default is `Unmanaged`), optional Alauda Service Mesh (`istio`) integration, and verification that the AmlCluster reports `Phase=Ready` and the workspace-controller deployment is available. No separate operator upload or `Workbench` custom resource is required.

## monitor

- [docs/en/monitor/bias_drift.mdx](docs/en/monitor/bias_drift.mdx): Guide to bias and drift monitoring with the TrustyAI Service, which collects, persists, and organizes model inference data for analysis. Covers accessing the TAS API and obtaining a token, uploading training and reference data, ingesting live KServe inference data, mapping column names to readable fields, and registering drift detection and bias metrics against reference and production traffic.
- [docs/en/monitor/index.mdx](docs/en/monitor/index.mdx): Chapter landing page for monitoring, covering logging and tracing for inference services, resource monitoring and dashboards, and model bias and drift monitoring. Notes that logging and resource monitoring serve inference service users while configuring dashboards requires the administrator view, and that bias and drift monitoring is provided by TrustyAI.
- [docs/en/monitor/logging_tracing/index.mdx](docs/en/monitor/logging_tracing/index.mdx): Section landing page for the Logging & Tracing area of Alauda AI's Monitoring & Ops module, rendering child topics via the `<Overview />` component as the entry point for log-related guides.
- [docs/en/monitor/logging_tracing/intro.mdx](docs/en/monitor/logging_tracing/intro.mdx): Introduces the Logging module that streams real-time container logs from inference service replica pods in MLOps/LLMOps/GenOps workflows, providing millisecond-latency visibility and built-in analysis tooling to accelerate debugging and incident response.
- [docs/en/monitor/logging_tracing/logging.mdx](docs/en/monitor/logging_tracing/logging.mdx): Walkthrough for the Logging tab on an Inference Service detail page: real-time streaming of pod logs with a Replica dropdown for switching between replica pods, an in-viewer Find feature that highlights matches in yellow with up/down navigation, and an Export button that downloads the current log buffer as a `.txt` file for offline analysis.
- [docs/en/monitor/resource_monitoring/add_monitor_dashboard.mdx](docs/en/monitor/resource_monitoring/add_monitor_dashboard.mdx): Walks through creating a custom Grafana-style monitoring dashboard from the Administrator View under Operations Center > Monitor > Dashboards: setting the AML folder, tagging the resource with the `cpaas.io/published: "true"` label via YAML, and adding a Native PromQL panel (example query computes CPU utilization as a ratio of `container_cpu_usage_seconds_total` to `kube_pod_container_resource_limits_cpu_cores`). After creation the dashboard is reachable from Alauda AI View > Observe > Dashboards via Switch.
- [docs/en/monitor/resource_monitoring/index.mdx](docs/en/monitor/resource_monitoring/index.mdx): Top-level landing page for the Resource Monitoring subsection of Monitoring & Ops, surfacing intro, functions, how-to, and troubleshooting child topics through `<Overview />`.
- [docs/en/monitor/resource_monitoring/intro.mdx](docs/en/monitor/resource_monitoring/intro.mdx): Introduces Resource Monitoring as the Monitoring & Ops component for tracking inference service resource utilization. Documents key usage limits: a 60-second minimum scrape interval, 7-day default historical retention, and dependencies on the Prometheus/VictoriaMetrics stack with node_exporter on workers and DCGM exporter on GPU nodes.
- [docs/en/monitor/resource_monitoring/monitor_dashboard_loading_stuck.mdx](docs/en/monitor/resource_monitoring/monitor_dashboard_loading_stuck.mdx): Workaround for inference service monitoring dashboards that hang on loading when AML is deployed on ACP 4.2.0 due to tightened default security policies. Instructs editing the `warlock` ClusterRole in `cpaas-system` to grant `update` on `components.aml.dev/amls/finalizers`, then restarting the warlock Deployment.
- [docs/en/monitor/resource_monitoring/monitor_pending_workload.mdx](docs/en/monitor/resource_monitoring/monitor_pending_workload.mdx): Walks through using the `VisibilityOnDemand` feature in Alauda Build of Kueue to inspect pending workloads in both `ClusterQueue` and `LocalQueue` via the `visibility.kueue.x-k8s.io/v1beta2` API, including example `FlowSchema`/`PriorityLevelConfiguration` for Kubernetes API Priority and Fairness throttling, RBAC bindings to `kueue-batch-admin-role` and `kueue-batch-user-role`, and a hands-on demo that creates `ResourceFlavor`, `ClusterQueue`, `LocalQueue`, and six suspended sample Jobs then queries `pendingworkloads` with `limit`/`offset` parameters.
- [docs/en/monitor/resource_monitoring/resource_monitoring.mdx](docs/en/monitor/resource_monitoring/resource_monitoring.mdx): Complete reference for the Resource Monitor tab on inference service detail pages, covering CPU/memory usage and utilization, GPU compute and VRAM metrics (unsupported under MPS deployment mode), vLLM-only token prompt/generation counters, and request metrics (response time avg/tp50/p90/p95, QPS, traffic). Includes step-by-step navigation, the 30-minute-to-7-day time range picker, calculation formulas, and operational thresholds such as the >90% sustained CPU scale-up cue and Kubernetes OOM behavior.

## overview

- [docs/en/overview/architecture.mdx](docs/en/overview/architecture.mdx): Architectural reference for the Alauda AI platform, organized by layer with component-by-component licensing and provenance. Catalogs the ACP layer (Alauda Builds of NVIDIA Device Plugin, HAMi, DCGM-Exporter, NPU Operator, Node Feature Discovery, NVIDIA DRA Driver, Volcano, Kueue, Milvus, PGVector), the AI Platform layer (Model Catalog, Kubeflow Model Registry, Datasets, Label Studio, FeatureForm, Workbench, Kubeflow Trainer v2, Kubeflow Pipelines, Guardrails, Evidently, MLflow), and the GenAI layer (KServe, vLLM, llm-d, Envoy AI Gateway as MaaS, fine-tuning with llama-factory, llm-compressor quantization, lm-evaluation-harness, Llama Stack, Langchain, Dify, MCP Servers, MLflow agent tracing, RAGAS).
- [docs/en/overview/glossary.mdx](docs/en/overview/glossary.mdx): Glossary of core Alauda AI terms, defining Large Language Model, Inference Service, Inference Runtime, and related platform concepts used throughout the documentation.
- [docs/en/overview/index.mdx](docs/en/overview/index.mdx): Entry-point landing page for the Alauda AI Overview section, delegating to child pages such as the architecture reference through the `<Overview />` component.
- [docs/en/overview/intro.mdx](docs/en/overview/intro.mdx): Introduces the Alauda AI platform as an MLOps and LLMOps best-practice toolkit for model storage and versioning, inference service release for large models, and codeless agent orchestration and AI application development. Frames the platform's audience (data scientists, ML engineers, application developers) and contrasts MLOps with LLMOps workflows.
- [docs/en/overview/quick_start.mdx](docs/en/overview/quick_start.mdx): Walks new users through a roughly 20-minute end-to-end Alauda AI tutorial: creating a namespace with the aml-namespace-editor/owner/viewer roles, bringing it under Alauda AI management, uploading a model such as Meta-Llama-3-8B-Instruct to the Model Repository, and publishing a vllm-cuda12.6-x86 inference service with HAMi NVIDIA GPU acceleration. Closes by invoking the deployed service via the in-console Experience feature for text generation.
- [docs/en/overview/release_notes.mdx](docs/en/overview/release_notes.mdx): Release notes for Alauda AI, listing new and optimized features, breaking changes, fixed issues, and known issues per release. The current release adds Model as a Service, a managed model-serving gateway built on Alauda Build of Envoy Gateway and Envoy AI Gateway with Authorino authentication, letting administrators publish inference services through subscriptions with token limits and usage reporting.

## plan

- [docs/en/plan/device_options/hami.mdx](docs/en/plan/device_options/hami.mdx): Introduces the Alauda Build of HAMi (Heterogeneous AI Computing Virtualization Middleware, formerly k8s-vGPU-scheduler), packaged as an all-in-one Helm chart for managing heterogeneous AI accelerators in a Kubernetes cluster and enabling GPU sharing across tasks. Links out to the dedicated HAMi external documentation site.
- [docs/en/plan/device_options/index.mdx](docs/en/plan/device_options/index.mdx): Section index for device options, linking the HAMi GPU virtualization and NVIDIA GPU device plugin guides used to expose accelerators to Alauda AI workloads.
- [docs/en/plan/device_options/pgpu.mdx](docs/en/plan/device_options/pgpu.mdx): Introduces the Alauda Build of the NVIDIA GPU Device Plugin, a Kubernetes DaemonSet that exposes per-node GPU counts, monitors GPU health, and enables GPU-enabled containers in the cluster. Links to the dedicated pgpu external documentation site for installation and configuration details.
- [docs/en/plan/index.mdx](docs/en/plan/index.mdx): Chapter landing page for planning, used before deployment to confirm component versions and architecture combinations in the supported configurations table, pick a validated model recipe, and choose a GPU or NPU option. The supported configurations table also indexes every component and links to its installation guide.
- [docs/en/plan/supported_configurations.md](docs/en/plan/supported_configurations.md): Support matrix listing the currently maintained Alauda AI versions, covering the current release and the most recent Stable release before it. Provides separate x86_64 and ARM architecture tables of supported Alauda Container Platform versions and component versions with their delivery type, plus notes on retired and replaced components.
- [docs/en/plan/validated_models/deepseek-v4-flash-w4a8.mdx](docs/en/plan/validated_models/deepseek-v4-flash-w4a8.mdx): Validated deployment recipe for DeepSeek-V4-Flash served as W4A8, a 256-expert MoE with MLA and DSA sparse attention and a native MTP speculative head at roughly 151 GB of weights. Validated on Ascend 910B4 with the vLLM-Ascend nightly engine through the InferNex surface at one instance with TP=8 and expert parallel, benchmarked through both the internal KServe ingress and the MaaS gateway.
- [docs/en/plan/validated_models/deepseek-v4-flash-w8a8.mdx](docs/en/plan/validated_models/deepseek-v4-flash-w8a8.mdx): Validated deployment recipe for DeepSeek-V4-Flash served as W8A8 at roughly 280 GB of weights across 70 shards. Because the weights exceed one node it was validated on Ascend 910B3 across two nodes as a single aggregated DP2 by TP=8 with EP16 service, adding a mooncake cross-rank KV store so both data-parallel ranks share prefix KV. Includes the ModelCar import and deployment steps.
- [docs/en/plan/validated_models/index.mdx](docs/en/plan/validated_models/index.mdx): Ready-to-deploy recipes for validated open-weight LLMs on Alauda AI, each deployed end-to-end on a real cluster and benchmarked. Covers the validated model list, runtime images, benchmark scenarios, the deployment procedure through the InferNex-Bridge into a hermes-router and vLLM-Ascend instances, caveats, and how to verify the ModelCar signature.
- [docs/en/plan/validated_models/minimax-m2.5-w8a8.mdx](docs/en/plan/validated_models/minimax-m2.5-w8a8.mdx): Validated deployment recipe for MiniMax-M2.5 served as W8A8, a 256-expert MoE with 62 layers of full attention and a native MTP head at roughly 230 GB of weights. Validated on Ascend 910B3 across two nodes as a single aggregated service, with instructions for staging weights on node-local hostPath volumes or importing the OCI ModelCar into a private registry.
- [docs/en/plan/validated_models/qwen3-32b.mdx](docs/en/plan/validated_models/qwen3-32b.mdx): Validated deployment recipe for Qwen3-32B, a standard dense model served in BF16 at roughly 62 GB across four cards. Validated on Ascend 910B4 with vLLM-Ascend v0.18.0 through the InferNex surface at TP=4 with two replicas, benchmarked in both the agg-base load-balancing spec and the agg-mc-kv cross-instance KV cache spec with KV-cache-aware routing.
- [docs/en/plan/validated_models/qwen3-6-27b-w8a8.mdx](docs/en/plan/validated_models/qwen3-6-27b-w8a8.mdx): Validated deployment recipe for Qwen3.6-27B served as W8A8, a Gated DeltaNet linear-attention hybrid with a native MTP speculative head at roughly 33 GB of INT8 weights. Validated on Ascend 910B4 with the vLLM-Ascend nightly engine through the InferNex surface at TP=4 with two replicas using the agg-base deployment spec.

## root

- [docs/en/index.mdx](docs/en/index.mdx): Root navigation page for the Alauda AI documentation site, rendering the top-level chapter listing.

## train

- [docs/en/train/components/index.mdx](docs/en/train/components/index.mdx): Section index for the components the Train chapter relies on, gathering the introduction and installation guides for Kueue, Volcano, and JobSet.
- [docs/en/train/components/jobset/index.mdx](docs/en/train/components/jobset/index.mdx): Component landing page for Alauda Build of JobSet, linking its introduction, installation guide, and quickstart.
- [docs/en/train/components/jobset/install.mdx](docs/en/train/components/jobset/install.mdx): Installs Alauda Build of JobSet on Alauda Container Platform through the Operator Hub. JobSet ships as a Helm-based operator: installing it deploys the jobset-operator controller, and creating a `JobSetOperatorCtl` instance then deploys the JobSet controller and webhook that reconcile `JobSet` custom resources.
- [docs/en/train/components/jobset/intro.mdx](docs/en/train/components/jobset/intro.mdx): Introduction to Alauda Build of JobSet, based on the Kubernetes SIG JobSet project, a Kubernetes-native API for managing a group of Jobs as a single unit. Describes the `jobset.x-k8s.io/v1alpha2` JobSet CRD and its ReplicatedJobs templates, offering a unified API for HPC and distributed AI/ML training workloads such as PyTorch, JAX, and TensorFlow.
- [docs/en/train/components/jobset/quickstart.mdx](docs/en/train/components/jobset/quickstart.mdx): Quickstart that runs a minimal JobSet demo to confirm the operator works and to introduce the most common fields of the `JobSet` custom resource. Covers prerequisites, a JobSet CR overview, running the demo, inspecting the created Jobs and pods, verifying completion, cleanup, and next steps.
- [docs/en/train/components/kueue/index.mdx](docs/en/train/components/kueue/index.mdx): Component landing page for Alauda Build of Kueue, linking its introduction and installation guide.
- [docs/en/train/components/kueue/install.mdx](docs/en/train/components/kueue/install.mdx): Installation guide for the Alauda Build of Kueue cluster plugin: download the package from the Customer Portal, upload it to ACP using the cluster plugin upload procedure, then deploy from `Administrator > Marketplace > Cluster Plugin` against the target cluster and verify with `kubectl get pods -n cpaas-system | grep kueue`. Also describes upgrading by uploading a new package and clicking `Upgrade` under the cluster's `Functional Components` page.
- [docs/en/train/components/kueue/intro.mdx](docs/en/train/components/kueue/intro.mdx): Introductory page positioning Alauda Build of Kueue as a Kubernetes-native quota-and-job-admission system that decides when jobs wait, start (pods are created), or are preempted (active pods deleted). Highlights that it integrates with the existing API server, scheduler, and cluster autoscaler rather than replacing them, and enforces all-or-nothing admission semantics for jobs.
- [docs/en/train/components/volcano/index.mdx](docs/en/train/components/volcano/index.mdx): Component landing page for Alauda support for Volcano, linking its introduction and installation guide.
- [docs/en/train/components/volcano/install.mdx](docs/en/train/components/volcano/install.mdx): Installs the Volcano cluster plugin retrieved from the Customer Portal. Covers downloading and uploading the cluster plugin, installing Volcano on the target cluster, upgrading between versions, and uninstalling.
- [docs/en/train/components/volcano/intro.mdx](docs/en/train/components/volcano/intro.mdx): Introduction to Alauda support for Volcano, which packages the Volcano CNCF project as a Cluster Plugin. Volcano is a cloud-native batch system providing gang scheduling, fair-share, and binpack policies required by distributed training and batch workloads, alongside the default Kubernetes scheduler, plus the `batch.volcano.sh/v1alpha1` VolcanoJob resource.
- [docs/en/train/guides/checkpointing-and-resuming.mdx](docs/en/train/guides/checkpointing-and-resuming.mdx): Guide to making Kubeflow Trainer v2 TrainJobs survive node reboots, evictions, preemptions, and OOM without losing hours of training. Explains that Trainer v2 has no checkpoint API of its own, what a checkpoint must contain, choosing storage that outlives the pod, making the runtime checkpoint-aware with numeric sorting, submitting a resumable TrainJob, and how crashes, preemptions, and deliberate pauses each resume.
- [docs/en/train/guides/fine-tune-and-pretrain-llms-on-ascend-npu.mdx](docs/en/train/guides/fine-tune-and-pretrain-llms-on-ascend-npu.mdx): Workbench-based recipes for full-parameter supervised fine-tuning and pretraining on arm64 nodes with Huawei Ascend NPUs, running training directly inside a single workbench container with multiple NPUs attached rather than through a VolcanoJob. Covers preparing the base model, Alpaca-style JSONL fine-tuning data and raw-text pretraining data, running the notebooks, key parameters, and output paths.
- [docs/en/train/guides/fine-tune-with-trainer-v2.mdx](docs/en/train/guides/fine-tune-with-trainer-v2.mdx): Guide to running supervised fine-tuning with LlamaFactory on Kubernetes using Kubeflow Trainer v2, which splits the job into a reusable `TrainingRuntime` holding the image, pipeline steps, and LlamaFactory config, and per-experiment `TrainJob` runs that override only the model, dataset, hyperparameters, and GPU resources. Covers building or using a prebuilt image, the example notebook, and Kueue scheduling.
- [docs/en/train/guides/fine-tuning-pipeline-with-mlflow-trustyai.mdx](docs/en/train/guides/fine-tuning-pipeline-with-mlflow-trustyai.mdx): Complete Kubeflow Pipeline that fine-tunes an LLM, tracks the run in MLflow, registers the model in the MLflow Model Registry, deploys it as a KServe InferenceService, evaluates it with a TrustyAI LMEvalJob, writes the evaluation numbers back to the same experiment, and cleans up the temporary serving resources. Wired to a KFP Recurring Run so it fires daily, building a train-and-evaluate history.
- [docs/en/train/guides/fine-tuning-using-notebooks.mdx](docs/en/train/guides/fine-tuning-using-notebooks.mdx): Guide to fine-tuning an LLM with LLaMA-Factory launched from an Alauda AI Workbench, where the notebook submits a VolcanoJob so GPU work runs on cluster nodes while iteration continues in JupyterLab. Covers creating a notebook or VSCode instance, preparing the base model, output placeholder, and dataset in HuggingFace or LLaMA-Factory format, the runtime image, submitting and managing the job, and experiment tracking.
- [docs/en/train/guides/gpu-slicing-with-dra.mdx](docs/en/train/guides/gpu-slicing-with-dra.mdx): Guide to requesting a slice of a GPU instead of a whole card using Dynamic Resource Allocation, the `resource.k8s.io/v1` API that reached GA in Kubernetes 1.34, with the Alauda Build of NVIDIA DRA Driver for GPUs. Contrasts DRA with the classic device-plugin model, describes the three objects in a slice request, and walks confirming the driver advertises slices through to running a sliced workload.
- [docs/en/train/guides/index.mdx](docs/en/train/guides/index.mdx): Section index for the training guides, providing end-to-end recipes for fine-tuning and pretraining LLMs on Alauda AI with a decision table that maps goals such as reusable templates, surviving preemption, or mixing training with online inference to the appropriate guide.
- [docs/en/train/guides/kubeflow-trainer-quick-start.md](docs/en/train/guides/kubeflow-trainer-quick-start.md): Minimal distributed PyTorch training setup on Alauda AI using Kubeflow Trainer v2, consisting of a custom runtime image, a `ClusterTrainingRuntime`, and an MNIST example notebook. Covers using the prebuilt torch-distributed image or building one from the supplied Containerfile.
- [docs/en/train/guides/preemptible-trainjobs-with-kueue.mdx](docs/en/train/guides/preemptible-trainjobs-with-kueue.mdx): Guide to letting training jobs share a cluster with online InferenceService workloads so inference is always protected while training borrows idle GPU and yields when inference reclaims quota. Covers the two-ClusterQueue cohort layout, Kueue WorkloadPriorityClasses, making the TrainJob preemption-safe with checkpointing, submitting the workloads, symmetric cohorts for namespace-level reservations, and verification.
- [docs/en/train/guides/training-hub-fine-tuning.mdx](docs/en/train/guides/training-hub-fine-tuning.mdx): Guide to the training_hub Python library, which wraps supervised fine-tuning, Orthogonal Subspace Fine-Tuning, LoRA and QLoRA, and continued pre-training behind single function calls that handle single-GPU, multi-GPU, and multi-node training uniformly. Covers requirements, data format, running the example notebooks, key parameters including automatic memory management, and multi-node setup.
- [docs/en/train/guides/training-runtimes.mdx](docs/en/train/guides/training-runtimes.mdx): Reference for the curated `TrainingRuntime` images available for Kubeflow Trainer v2, each bundling a specific PyTorch and accelerator stack so users can submit TrainJobs without rebuilding. Lists the available runtimes for NVIDIA GPU with CUDA and Huawei Ascend NPU with CANN, guidance on picking one, applying a TrainingRuntime, device resource requests, image caveats, and building your own.
- [docs/en/train/index.mdx](docs/en/train/index.mdx): Chapter landing page for training, covering fine-tuning and pretraining, training runtime images, checkpointing and resuming, GPU slicing, and quota and scheduling governance. Notes that Kueue handles queueing, quotas, and preemption while Volcano and JobSet handle batch orchestration, and that experiment tracking uses MLflow from the Develop chapter.
- [docs/en/train/quota_scheduling/config_quotas.mdx](docs/en/train/quota_scheduling/config_quotas.mdx): Administrator procedure for setting up Kueue quotas by creating a ClusterQueue with resourceGroups covering CPU/memory/pods plus GPU resources (nvidia.com/gpualloc, nvidia.com/total-gpucores, nvidia.com/total-gpumem for Alauda Build of Hami, or nvidia.com/gpu for the Alauda Build of NVIDIA GPU Device Plugin), defining ResourceFlavor objects keyed on nodeLabels like nvidia.com/gpu.product=Tesla-T4 or NVIDIA-A30, and binding them to namespaces with LocalQueue objects (including a default queue that auto-labels jobs with kueue.x-k8s.io/queue-name).
- [docs/en/train/quota_scheduling/fair_sharing.mdx](docs/en/train/quota_scheduling/fair_sharing.mdx): Explains how fair sharing in Alauda Build of Kueue distributes borrowable (unused nominal) quota across tenants in a cohort using ClusterQueue weights configured under spec.fairSharing.weight. Notes that lower share values are admitted first and preempted last, the default weight is 1, and a weight of 0 represents an infinite share value that makes the queue always the first preemption target.
- [docs/en/train/quota_scheduling/gang_scheduling.mdx](docs/en/train/quota_scheduling/gang_scheduling.mdx): Describes Kueue's timeout-based gang (all-or-nothing) scheduling, which suspends groups of related jobs until the cluster can guarantee capacity for the whole gang, preventing GPU under-utilization, resource segmentation, and deadlocks. Notes that the feature is enabled by default and that administrators can adjust the timeout or disable it through the deployment form parameters of the Alauda Build of Kueue cluster plugin.
- [docs/en/train/quota_scheduling/index.mdx](docs/en/train/quota_scheduling/index.mdx): Section index for quota and scheduling, linking the guides that configure Kueue quotas, cohorts, fair sharing, RBAC, and Ascend NPU quota for training workloads.
- [docs/en/train/quota_scheduling/npu_quota.mdx](docs/en/train/quota_scheduling/npu_quota.mdx): Guide to governing Ascend NPU quota with Alauda Build of Kueue. Kueue treats `huawei.com/Ascend910` and similar device-plugin resources as ordinary countable resources, so the same ClusterQueue, ResourceFlavor, and LocalQueue model used for CPU, memory, and NVIDIA GPUs applies unchanged. Covers identifying the NPU node label, creating the Kueue objects, submitting a job, and observing admission.
- [docs/en/train/quota_scheduling/setup_rbac.mdx](docs/en/train/quota_scheduling/setup_rbac.mdx): Explains the two built-in ClusterRoles created when installing Alauda Build of Kueue: `kueue-batch-admin-role` (managing ClusterQueues, Queues, Workloads, and ResourceFlavors) and `kueue-batch-user-role` (managing Jobs and viewing Queues/Workloads). Provides ready-to-apply ClusterRoleBinding and RoleBinding YAML examples for binding these roles to a batch administrator (`admin@cpaas.com`) cluster-wide and a batch user (`team-a-owner@cpaas.com`) scoped to a `team-a` namespace.
- [docs/en/train/quota_scheduling/using_cohorts.mdx](docs/en/train/quota_scheduling/using_cohorts.mdx): Describes how to group `ClusterQueue` objects into cohorts via the `.spec.cohort` field so they can share borrowable resources (unused nominal quota) across the group, enabling fair sharing and better utilization for related teams or workloads. Notes that omitting `spec.cohort` excludes a cluster queue from cohort-level borrowing and that cohorts can be used to enforce group-level resource quotas.

## upgrade

- [docs/en/upgrade/index.mdx](docs/en/upgrade/index.mdx): Chapter landing page for upgrade and uninstall, linking the version upgrade procedure, the Knative Operator migration guide, the Workbench upgrade guide, and the uninstall procedure.
- [docs/en/upgrade/migrating-to-knative-operator.mdx](docs/en/upgrade/migrating-to-knative-operator.mdx): Migration path from the legacy Alauda AI Model Serving operator (1.x series) to the upstream Knative Operator (2.x series) for inference-service serverless capability. Instructs operators to set spec.knativeServing.managementState to Removed on the default Alauda AI instance, then install Knative Operator and create a KnativeServing instance via the Marketplace per the Enabling Knative Functionality guide. Notes that AI 1.x deployments must also upgrade Alauda AI to 2.x.
- [docs/en/upgrade/uninstall.mdx](docs/en/upgrade/uninstall.mdx): Step-by-step uninstall procedure for Alauda AI 2.8 and its components. Components managed by the `default` `AmlCluster` are uninstalled by setting `managementState` to `Removed` (for `Managed` components) or to `Unmanaged` then uninstalling the Operator in OperatorHub (for `SharedManaged` components); components not managed by the AmlCluster are uninstalled from OperatorHub or Cluster Plugins. The core Alauda AI is removed by deleting the `default` `AmlCluster` instance, then uninstalling the Alauda AI Operator. Warns that data-related resources remain in the `kubeflow` and `aml-maas` namespaces after uninstallation, that operator instances should be deleted before the Operator, and that uninstalling does not remove CRDs.
- [docs/en/upgrade/upgrade-from-previous-version.mdx](docs/en/upgrade/upgrade-from-previous-version.mdx): Upgrades Alauda AI from 2.3.x to 2.8.x, migrating from the 2.3 deployment model to the single-cluster `AmlCluster` model. Covers preparing and uploading the 2.8 package set, preserving existing resources, removing the old Cluster Plugin forms of Alauda AI Essentials, LeaderWorkerSet, Workbench, and MLflow, running `global-install.sh` and `migrate-roles.sh` on the global cluster, upgrading the Alauda AI Operator (which auto-upgrades the `default` `AmlCluster` and enables PostgreSQL, Redis, Authorino, KServe, and LWS), manually enabling optional components (MLflow, Workbench, Serving Runtime), and post-upgrade verification.
- [docs/en/upgrade/workbench.mdx](docs/en/upgrade/workbench.mdx): Guide to upgrading Workbench in Alauda AI 2.8.x, where Workbench is managed by Alauda AI through the `default` `AmlCluster`. Covers migrating from the Workbench Cluster Plugin while preserving user PVCs, retained WorkspaceKind resources, and the aml-workbench-config ConfigMap, enabling `spec.components.workbench.managementState: Managed` (no separate `Workbench` custom resource), migrating from Kubeflow Notebook, and preserving data.