Skip to content

No output from az datafactory activity-run query-by-pipeline-run #8940

@rgant

Description

@rgant

Describe the bug

No output from az datafactory activity-run query-by-pipeline-run despite data in the debug log.

Related command

az datafactory activity-run query-by-pipeline-run \
    --factory-name "$DATA_FACTORY_NAME" \
    --last-updated-after "2025-07-01T00:36:44.3345758Z" 
    --last-updated-before "2025-07-16T00:49:48.3686473Z" \
    --resource-group "$RESOURCE_GROUP_NAME" \
    --run-id "$RUN_ID" \
    --debug

Errors

No error message, no output at all (other than the debug information).

Issue script & Debug output

Command Debug Output
cli.knack.cli: Command arguments: ['datafactory', 'activity-run', 'query-by-pipeline-run', '--factory-name', '$DATA_FACTORY_NAME', '--last-updated-after', '2025-07-01T00:36:44.3345758Z', '--last-updated-before', '2025-07-16T00:49:48.3686473Z', '--resource-group', '$RESOURCE_GROUP_NAME', '--run-id', '$RUN_ID', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x10269b240>, <function OutputProducer.on_global_arguments at 0x1028ac2c0>, <function CLIQuery.on_global_arguments at 0x1028c9a80>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'datafactory': ['azext_datafactory']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: Total (0)                 0.000         0         0
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: datafactory               0.002        16        79  /Users/rgant/.azure/cliextensions/datafactory
cli.azure.cli.core: Total (1)                 0.002        16        79  
cli.azure.cli.core: Loaded 16 groups, 79 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : datafactory activity-run query-by-pipeline-run
cli.azure.cli.core: Command table: datafactory activity-run query-by-pipeline-run
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x103e7bb00>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/Users/rgant/.azure/commands/2025-07-04.10-23-55.datafactory_activity-run_query-by-pipeline-run.81597.log'.
az_command_data_logger: command args: datafactory activity-run query-by-pipeline-run --factory-name {} --last-updated-after {} --last-updated-before {} --resource-group {} --run-id {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x103ecafc0>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x103ecb060>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x103ecb1a0>, <function register_upcoming_breaking_change_info.<locals>.update_breaking_change_info at 0x103ecb240>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x1028ac360>, <function CLIQuery.handle_query_parameter at 0x1028c9b20>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x103ecb100>]
az_command_data_logger: extension name: datafactory
az_command_data_logger: extension version: 1.0.4
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=DataFactoryManagementClient
cli.azure.cli.core.auth.persistence: build_persistence: location='/Users/rgant/.azure/msal_token_cache.json', encrypt=False
cli.azure.cli.core.auth.binary_cache: load: /Users/rgant/.azure/msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: Initializing with Entra authority: https://login.microsoftonline.com/$GUID
msal.authority: openid_config("https://login.microsoftonline.com/$GUID/v2.0/.well-known/openid-configuration") = {'token_endpoint': 'https://login.microsoftonline.com/$GUID/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/$GUID/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/$GUID/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/$GUID/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/$GUID/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/$GUID/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/$GUID/kerberos', 'tenant_region_scope': 'NA', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'}
msal.application: Broker enabled? None
cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token_info: scopes=('https://management.core.windows.net//.default',), options={}
cli.azure.cli.core.auth.msal_credentials: UserCredential.acquire_token: scopes=['https://management.core.windows.net//.default'], claims_challenge=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: 91219b92-1da4-4d5a-a911-b51972903362
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.DataFactory/factories/$DATA_FACTORY_NAME/pipelineruns/$RUN_ID/queryActivityruns?api-version=2018-06-01'
cli.azure.cli.core.sdk.policies: Request method: 'POST'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies:     'Content-Type': 'application/json'
cli.azure.cli.core.sdk.policies:     'Content-Length': '103'
cli.azure.cli.core.sdk.policies:     'Accept': 'application/json'
cli.azure.cli.core.sdk.policies:     'x-ms-client-request-id': '8447822a-58e2-11f0-9db8-9663bd5d8b90'
cli.azure.cli.core.sdk.policies:     'CommandName': 'datafactory activity-run query-by-pipeline-run'
cli.azure.cli.core.sdk.policies:     'ParameterSetName': '--factory-name --last-updated-after --last-updated-before --resource-group --run-id --debug'
cli.azure.cli.core.sdk.policies:     'User-Agent': 'AZURECLI/2.75.0 (HOMEBREW) azsdk-python-core/1.31.0 Python/3.12.11 (macOS-15.5-arm64-arm-64bit)'
cli.azure.cli.core.sdk.policies:     'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: {"lastUpdatedAfter": "2025-07-01T00:36:44.334575Z", "lastUpdatedBefore": "2025-07-16T00:49:48.368647Z"}
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "POST /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.DataFactory/factories/$DATA_FACTORY_NAME/pipelineruns/$RUN_ID/queryActivityruns?api-version=2018-06-01 HTTP/1.1" 200 2533
cli.azure.cli.core.sdk.policies: Response status: 200
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies:     'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies:     'Content-Length': '2533'
cli.azure.cli.core.sdk.policies:     'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies:     'Expires': '-1'
cli.azure.cli.core.sdk.policies:     'x-ms-correlation-request-id': '37d3b5b5-c212-4e4c-bf9d-6e9c0e93cf1d'
cli.azure.cli.core.sdk.policies:     'x-ms-operation-identifier': 'tenantId=$GUID,objectId=545b8fc4-3b92-40d1-9998-2e5b867a9e09/southcentralus/744b41b2-3356-45f9-b459-beb97ea7253b'
cli.azure.cli.core.sdk.policies:     'x-ms-ratelimit-remaining-subscription-reads': '249'
cli.azure.cli.core.sdk.policies:     'x-ms-ratelimit-remaining-subscription-global-reads': '3749'
cli.azure.cli.core.sdk.policies:     'x-ms-request-id': '37d3b5b5-c212-4e4c-bf9d-6e9c0e93cf1d'
cli.azure.cli.core.sdk.policies:     'x-ms-routing-request-id': 'SOUTHCENTRALUS:20250704T142355Z:37d3b5b5-c212-4e4c-bf9d-6e9c0e93cf1d'
cli.azure.cli.core.sdk.policies:     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.sdk.policies:     'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies:     'X-Cache': 'CONFIG_NOCACHE'
cli.azure.cli.core.sdk.policies:     'X-MSEdge-Ref': 'Ref A: 2C52F586AAB843368F6BB368ED094620 Ref B: BL2AA2011005062 Ref C: 2025-07-04T14:23:55Z'
cli.azure.cli.core.sdk.policies:     'Date': 'Fri, 04 Jul 2025 14:23:55 GMT'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {
  "value": [
    {
      "activityRunEnd": "2025-07-04T13:23:06.3921486Z",
      "activityName": "GetSFTPMetadata",
      "activityRunStart": "2025-07-04T13:20:19.6258242Z",
      "activityType": "GetMetadata",
      "durationInMs": 166766,
      "retryAttempt": null,
      "error": {
        "errorCode": "3500",
        "message": "Field 'exists' failed with error: 'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to get file attributes, Sftp server '$SFTP_SERVER_HOSTNAME', file path '.'. Please make sure Sftp server is not throttling ADF Service or Integration Runtime (Self-hosted).,Source=Microsoft.DataTransfer.ClientLibrary.SftpConnector,''Type=Renci.SshNet.Common.SshConnectionException,Message=Client not connected.,Source=Renci.SshNet,'.",
        "failureType": "UserError",
        "target": "GetSFTPMetadata",
        "details": []
      },
      "activityRunId": "83d09a79-20b9-42ac-a642-2ad3006e0854",
      "iterationHash": "",
      "input": {
        "dataset": {
          "referenceName": "$DATASET_NAME",
          "type": "DatasetReference"
        },
        "fieldList": [
          "exists",
          "itemName",
          "itemType",
          "lastModified"
        ]
      },
      "linkedServiceName": "",
      "output": {
        "effectiveIntegrationRuntime": "azure-ir-vnet (East US)",
        "executionDuration": 160,
        "durationInQueue": {
          "integrationRuntimeQueue": 0
        },
        "billingReference": {
          "activityType": "PipelineActivity",
          "billableDuration": [
            {
              "meterType": "AzureIR",
              "duration": 0.049999999999999996,
              "unit": "Hours"
            }
          ]
        }
      },
      "userProperties": {},
      "pipelineName": "$PIPELINE_NAME",
      "pipelineRunId": "$RUN_ID",
      "status": "Failed",
      "recoveryStatus": "None",
      "integrationRuntimeNames": [
        "azure-ir-vnet"
      ],
      "executionDetails": {
        "integrationRuntime": [
          {
            "name": "azure-ir-vnet",
            "type": "Managed",
            "location": "East US"
          }
        ]
      },
      "id": "/SUBSCRIPTIONS/$SUBSCRIPTION_ID/RESOURCEGROUPS/$RESOURCE_GROUP_NAME/PROVIDERS/MICROSOFT.DATAFACTORY/FACTORIES/$DATA_FACTORY_NAME/pipelineruns/$RUN_ID/activityruns/83d09a79-20b9-42ac-a642-2ad3006e0854"
    }
  ]
}
cli.knack.cli: Event: CommandInvoker.OnTransformResult [<function _resource_group_transform at 0x103ec8400>, <function _x509_from_base64_to_hex_transform at 0x103ec84a0>]
cli.knack.cli: Event: CommandInvoker.OnFilterResult []

cli.knack.cli: Event: Cli.SuccessfulExecute []
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x103e7bd80>]
az_command_data_logger: exit code: 0
cli.__main__: Command ran in 0.728 seconds (init: 0.067, invoke: 0.661)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 4112 in cache file under /Users/rgant/.azure/telemetry/20250704102356055
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/opt/homebrew/Cellar/azure-cli/2.75.0/libexec/bin/python /opt/homebrew/Cellar/azure-cli/2.75.0/libexec/lib/python3.12/site-packages/azure/cli/telemetry/__init__.py /Users/rgant/.azure /Users/rgant/.azure/telemetry/20250704102356055"
telemetry.process: Return from creating process 81602
telemetry.main: Finish creating telemetry upload process.

Some details have been replaced by $VARIABLES that I felt were sensitive.

Expected behavior

I expected to have some output showing the error that happened.

Environment Summary

$ az --version
azure-cli                         2.75.0

core                              2.75.0
telemetry                          1.1.0

Extensions:
datafactory                        1.0.4
init                               0.1.0

Dependencies:
msal                            1.33.0b1
azure-mgmt-resource               23.3.0

Python location '/opt/homebrew/Cellar/azure-cli/2.75.0/libexec/bin/python'
Config directory '/Users/rgant/.azure'
Extensions directory '/Users/rgant/.azure/cliextensions'

Python (Darwin) 3.12.11 (main, Jun  3 2025, 15:41:47) [Clang 17.0.0 (clang-1700.0.13.3)]

Legal docs and information: aka.ms/AzureCliLegal


Your CLI is up-to-date.

Additional context

Possibly related is that I frequently cannot get --querys to work. I need to call jq myself.

az deployment sub show \
            --name "$DEPLOYMENT_NAME" \
            --query "properties.outputs" \
            --output json | \
        jq -r 'to_entries[] | "\(.key)=\(.value.value)"'

Metadata

Metadata

Assignees

No one assigned

    Labels

    Auto-AssignAuto assign by botData FactoryService AttentionThis issue is responsible by Azure service team.bugThis issue requires a change to an existing behavior in the product in order to be resolved.customer-reportedIssues that are reported by GitHub users external to the Azure organization.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions