keyvualt timeouts

[scottmcburney]

We are now forced to use an HSM keyvault, which has cut access limits in HALF compared to a regular keyvault. This is causing throttling problems when signing large quantities of files.

It seems that azure sign tool does not handle throttling well, as we are having signing throttle a lot during our pipelines. I'm trying to address the problem as best I can from our pipelines, but it still happens a LOT. My wish is that when throttling happens, azuresigntool could "pause" itself for an amount of time (30 seconds? 1 minute?) and retry.

Status: 429 ()

Content:
{"error":{"code":"Throttled","message":"Request was not processed because too many requests were received. Reason: VaultRequestTypeLimitReached"}}

Unhandled exception. System.AggregateException: One or more errors occurred. (Error calling Key Vault)
---> System.Security.Cryptography.CryptographicException: Error calling Key Vault
---> Azure.RequestFailedException: Service request failed.
at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken)
at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest[TContent,TResult](RequestMethod method, TContent content, Func1 resultFactory, CancellationToken cancellationToken, String[] path) at Azure.Security.KeyVault.Keys.Cryptography.RemoteCryptographyClient.Sign(SignatureAlgorithm algorithm, Byte[] digest, CancellationToken cancellationToken) at Azure.Security.KeyVault.Keys.Cryptography.CryptographyClient.Sign(SignatureAlgorithm algorithm, Byte[] digest, CancellationToken cancellationToken) at RSAKeyVaultProvider.RSAKeyVault.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/RSAKeyVaultProvider/RSAKeyVault.cs:line 45 --- End of inner exception stack trace --- at RSAKeyVaultProvider.RSAKeyVault.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/RSAKeyVaultProvider/RSAKeyVault.cs:line 45 at AzureSign.Core.AuthenticodeKeyVaultSigner.SignCallback(IntPtr pCertContext, IntPtr pvExtra, UInt32 algId, Byte[] pDigestToSign, UInt32 dwDigestToSign, CRYPTOAPI_BLOB& blob) in /_/src/AzureSign.Core/AuthenticodeKeyVaultSigner.cs:line 239 at AzureSign.Core.Interop.mssign32.SignerSignEx3(SignerSignEx3Flags dwFlags, SIGNER_SUBJECT_INFO* pSubjectInfo, SIGNER_CERT* pSignerCert, SIGNER_SIGNATURE_INFO* pSignatureInfo, IntPtr pProviderInfo, SignerSignTimeStampFlags dwTimestampFlags, Byte* pszTimestampAlgorithmOid, Char* pwszHttpTimeStamp, IntPtr psRequest, Void* pSipData, IntPtr* ppSignerContext, IntPtr pCryptoPolicy, SIGN_INFO* pSignInfo, IntPtr pReserved) at AzureSign.Core.AuthenticodeKeyVaultSigner.SignFile(ReadOnlySpan1 path, ReadOnlySpan1 description, ReadOnlySpan1 descriptionUrl, Nullable1 pageHashing, ILogger logger) in /_/src/AzureSign.Core/AuthenticodeKeyVaultSigner.cs:line 88 at AzureSignTool.SignCommand.<>c__DisplayClass104_1.<OnExecuteAsync>b__2(String filePath, ParallelLoopState pls, ValueTuple2 state) in //src/AzureSignTool/SignCommand.cs:line 335
at System.Threading.Tasks.Parallel.<>c__DisplayClass44_02.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion) --- End of stack trace from previous location --- at System.Threading.Tasks.Parallel.<>c__DisplayClass44_02.b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
at System.Threading.Tasks.TaskReplicator.Replica1.ExecuteAction(Boolean& yieldedBeforeCompletion) at System.Threading.Tasks.TaskReplicator.Replica.Execute() --- End of inner exception stack trace --- at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction1 action, ParallelOptions options, Boolean stopOnFirstFailure)
at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner1 source, ParallelOptions parallelOptions, Action1 simpleBody, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally)
--- End of stack trace from previous location ---
at System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection exceptions, CancellationToken cancelToken, Exception otherException)
at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner1 source, ParallelOptions parallelOptions, Action1 simpleBody, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Action3 bodyWithStateAndIndex, Func4 bodyWithStateAndLocal, Func5 bodyWithEverything, Func1 localInit, Action1 localFinally)
at System.Threading.Tasks.Parallel.ForEach[TSource,TLocal](IEnumerable1 source, ParallelOptions parallelOptions, Func1 localInit, Func4 body, Action1 localFinally)
at AzureSignTool.SignCommand.OnExecuteAsync(CommandLineApplication app, IConsole console) in //src/AzureSignTool/SignCommand.cs:line 287
at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.InvokeAsync(MethodInfo method, Object instance, Object[] arguments)
at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context, CancellationToken cancellationToken)
at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<b__0>d.MoveNext()
--- End of stack trace from previous location ---
at McMaster.Extensions.CommandLineUtils.CommandLineApplication.ExecuteAsync(String[] args, CancellationToken cancellationToken)
at McMaster.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
at AzureSignTool.Program.Main(String[] args) in /_/src/AzureSignTool/Program.cs:line 26