Skip to content

[BUG] AppEnvironment pod template misses inject-flyte-secrets label/annotations and env vars #7600

Description

@flabbergastedbd

Summary

AppEnvironment deployed via flyte deploy produces a knative Service whose revision pod template is missing the flyte-managed pod customization that the TaskAction path injects. Two symptoms:

  1. secrets=[flyte.Secret(...)] is serialized into SecurityContext.secrets, but the pod gets neither the inject-flyte-secrets: "true" label (gates the flyte-binary-webhook admission webhook via objectSelector.matchLabels) nor the flyte.secrets/s* annotations from secrets.MarshalSecretsToMapStrings(...) (tell the webhook which keys to mount). Secret env vars never reach the pod.
  2. FLYTE_INTERNAL_EXECUTION_PROJECT / FLYTE_INTERNAL_EXECUTION_DOMAIN env vars are absent. flyte.init_in_cluster() reads project/domain from those; without them, in-app Run.get(...) builds malformed admin URLs of the form //RUN_ID (404) and with_runcontext(...).run(...) is rejected by require_project_and_domain.

Root cause

app/internal/k8s/app_client.go::buildKService populates only autoscaling annotations on Template.ObjectMeta and copies the user's container env verbatim. It does not perform the equivalent of executor/pkg/plugin/task_exec_metadata.go:81-94 (secret label + scope labels + MarshalSecretsToMapStrings annotations) or flyteplugins/.../flytek8s/k8s_resource_adds.go::GetExecutionEnvVars (FLYTE_INTERNAL_EXECUTION_* env vars). All inputs are available via app.GetMetadata().GetId().

Confirmed present on v2.0.25 (latest tag).

Reproduction

import flyte
from flyte.app import AppEnvironment

app_env = AppEnvironment(
    name="example-app", image=...,
    secrets=[flyte.Secret(key="my_api_key", as_env_var="MY_API_KEY")],
)

@app_env.server
def _serve():
    import os
    from flyte.remote import Run
    flyte.init_in_cluster()
    print(os.environ.get("MY_API_KEY"))                          # None
    print(os.environ.get("FLYTE_INTERNAL_EXECUTION_PROJECT"))    # None
    Run.get(name="EXISTING_RUN_ID")
    # connectrpc.errors.ConnectError: run not found: //EXISTING_RUN_ID

kubectl get pod APP_POD_NAME -o jsonpath='{.metadata.labels}' shows no inject-flyte-secrets; env lacks FLYTE_INTERNAL_EXECUTION_PROJECT / FLYTE_INTERNAL_EXECUTION_DOMAIN. The same cluster's task pods have both.

Environment

flyte python SDK 2.2.2, flyte-binary 0.2.0 (PR #7279 applied), knative-serving 1.18.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions