Test-AzResourceGroupDeployment does not return validation output or error properly

[AlanWheelwright]

Description

Test-AzResourceGroupDeployment does not return a powershell object of any kind and it's result is not capturable into a powershell variable for some reason.

$TestResult = Test-AzResourceGroupDeployment
Will result in $TestResult containing nothing when the validation test was successful even when set to verbose and a verbose output is displayed:
"VERBOSE: 12:44:15 PM - Template is valid."

$TestResult = Test-AzResourceGroupDeployment
Will result in $TestResult containing a single return character when the validation test was unsuccessful but verbose, no matter what the error or how many.

Even attempting to direct the error output into a variable via -ErrorVariable
Test-AzResourceGroupDeployment -ErrorVariable $TemplateValidationErrorVariable

results in $TemplateValidationErrorVariable being empty.

Issue script & Debug output

$ArmTemplateValidationParameters = @{
        ResourceGroupName = $TargetResourceGroupName
        TemplateFile =  "$FullTemplatePath"
        #Standard Parameters
        Mode = 'Incremental'
        Verbose = $true
        ErrorAction = 'Stop'
        WarningAction = 'Stop'
        ErrorVariable = $TemplateValidationError
    }
    Try {$ValidationResult = Test-AzResourceGroupDeployment @ArmTemplateValidationParameters
    } Catch {
        Write-Error "Template is Invalid!"
        $TemplateValidationSuccess = $false
    }

Environment data

Name                           Value
----                           -----
PSVersion                      7.5.0
PSEdition                      Core
GitCommitId                    7.5.0
OS                             Microsoft Windows 10.0.22631
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     4.0.2                 Az.Accounts                         {Add-AzEnvironment, Clear-AzConfig, Clear-AzContext, Clear-AzDefault…}
Script     9.1.0                 Az.Compute                          {Add-AzImageDataDisk, Add-AzVhd, Add-AzVMAdditionalUnattendContent, Add-AzVMDataDisk…}
Script     7.9.0                 Az.Resources                        {Export-AzResourceGroup, Export-AzTemplateSpec, Get-AzDenyAssignment, Get-AzDeployment…}

Error output

HistoryId: 574

Message        : Invalid character after parsing property name. Expected ':' but got: C. Path 'resources[0].type', line 10, position 9.
StackTrace     :    at Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.ResourceManagerCmdletBase.HandleException(ExceptionDispatchInfo
                 capturedException)
                    at Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.ResourceManagerCmdletBase.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : Newtonsoft.Json.JsonReaderException
InvocationInfo : {Test-AzResourceGroupDeployment}
Line           :     $ValidationResult = $(Test-AzResourceGroupDeployment @ArmTemplateValidationParameters)

Position       : At line:23 char:27
                 + … nResult = $(Test-AzResourceGroupDeployment @ArmTemplateValidationPara …
                 +               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 574

Message        : Template is Invalid!
StackTrace     :
Exception      : Microsoft.PowerShell.Commands.WriteErrorException
InvocationInfo : {If ($null -ne $TemplateContent){
                     #The template variable isn't empty, so validate the template file
                     $TemplateImportSuccess = $true
                     $ArmTemplateValidationParameters = @{
                         ResourceGroupName = $TargetResourceGroupName
                         TemplateFile =  "$FullExportedTemplatePath"
                         #Standard Parameters
                         Mode = 'Incremental'
                         Verbose = $true
                         ErrorAction = 'Stop'
                         WarningAction = 'Stop'
                         ErrorVariable = $TemplateValidationError
                     }
                     Try {$ValidationResult = Test-AzResourceGroupDeployment @ArmTemplateValidationParameters} Catch {
                         Write-Error "Template is Invalid!"
                         $TemplateValidationSuccess = $false
                         $TemplateImportSuccess = $false
                     }

                     #Use the validation result to determine the template validity
                     $ValidationResult = $(Test-AzResourceGroupDeployment @ArmTemplateValidationParameters)

                     IF ($(Test-AzResourceGroupDeployment @ArmTemplateValidationParameters) -match $TemplateValidationResultFailureRegexPattern) {
                         Write-Error "Template is Invalid Regex!"
                         $TemplateImportSuccess = $false
                     }

                 }}
Line           :
Position       :
HistoryId      : 574

Message        : Invalid character after parsing property name. Expected ':' but got: C. Path 'resources[0].type', line 10, position 9.
StackTrace     :    at Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.ResourceManagerCmdletBase.HandleException(ExceptionDispatchInfo
                 capturedException)
                    at Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.ResourceManagerCmdletBase.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : Newtonsoft.Json.JsonReaderException
InvocationInfo : {Test-AzResourceGroupDeployment}
Line           :     Try {$ValidationResult = Test-AzResourceGroupDeployment @ArmTemplateValidationParameters

Position       : At line:15 char:30
                 + … ionResult = Test-AzResourceGroupDeployment @ArmTemplateValidationPara …
                 +               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 574

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/deployments-owners.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/deployments-owners.

[o-l-a-v]

Just noticed it does not throw when it finds an error. One can check the output of the cmdlet though, something like this:

$Validate = Test-AzResourceGroupDeployment @Splat
if ($Validate.'Code' -eq 'InvalidTemplate') {
    $Validate | ConvertTo-Json
    Exit 1
}

My output looked like this:

{
  "Code": "InvalidTemplate",
  "Message": "Deployment template validation failed: 'The template resource 'dnsZone_roleAssignment' at line '226' and column '35' is not valid: The language expression property 'dnsZoneContributor' doesn't exist, available properties are 'keyVaultCertificatesOfficer, dnzZoneContributor'.. Please see https://aka.ms/arm-functions for usage details.'.",
  "Target": null,
  "Details": null
}

I'd expect the cmdlet to throw if template was not validated as legit though.

And when it does validate as legit, I don't expect it to throw an empty warning: #27888