{CI} add azdev scan (medium confidence) task

[evelyn-ys]

By default azdev scan will only use high confidence level patterns, this PR add additional task to scan medium confidence level patterns with warnings. Since there're possibilities of false positives, so it's failure won't block PR merge

---

This checklist is used to make sure that common guidelines for a pull request are followed.

Related command

General Guidelines

• Have you run azdev style <YOUR_EXT> locally? (pip install azdev required)
• Have you run python scripts/ci/test_index.py -q locally? (pip install wheel==0.30.0 required)
• My extension version conforms to the Extension version schema

For new extensions:

• My extension description/summary conforms to the Extension Summary Guidelines.

About Extension Publish

There is a pipeline to automatically build, upload and publish extension wheels.
Once your pull request is merged into main branch, a new pull request will be created to update src/index.json automatically.
You only need to update the version information in file setup.py and historical information in file HISTORY.rst in your PR but do not modify src/index.json.

[azure-client-tools-bot-prd]

️✔️Azure CLI Extensions Breaking Change Test

️✔️Non Breaking Changes

[azure-client-tools-bot-prd]

Hi @evelyn-ys,
Please write the description of changes which can be perceived by customers into HISTORY.rst.
If you want to release a new extension version, please update the version in setup.py as well.

[github-actions]

🚫All pull requests will be blocked to merge until Jan 6, 2025 due to CCOA

[yonzhan]

Thank you for your contribution! We will review the pull request and get back to you soon.

[github-actions]

• For more info about extension versioning, please refer to Extension version schema

[github-actions]

CodeGen Tools Feedback Collection

Thank you for using our CodeGen tool. We value your feedback, and we would like to know how we can improve our product. Please take a few minutes to fill our codegen survey

[wangzelin007]

Suggested change

	- job: AzdevScanProModifiedExtensions
	displayName: "azdev scan - Pro on Modified Extensions"
	dependsOn: AzdevScanModifiedExtensions
	condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
	continueOnError: true
	pool:
	name: 'pool-ubuntu-2004'
	steps:
	- task: UsePythonVersion@0
	displayName: 'Use Python 3.11'
	inputs:
	versionSpec: 3.11
	- template: .azure-pipelines/templates/azdev_setup.yml
	- bash: |
	#!/usr/bin/env bash
	set -ev
	source ./env/bin/activate
	git fetch origin --depth=1 $(System.PullRequest.TargetBranch)
	declare -A secret_files
	for FILE in `git diff --name-only --diff-filter=AM origin/$(System.PullRequest.TargetBranch)` ; do
	detected=$(azdev scan --confidence-level MEDIUM -f $FILE --continue-on-failure| python -c "import sys, json; print(json.load(sys.stdin)['secrets_detected'])")
	if [ $detected == 'True' ]; then
	printf "\033[0;31mSecrets detected from %s, Please remove or replace it. You can run 'azdev scan'/'azdev mask' locally to fix.\033[0m" "$FILE"
	secret_files+=$FILE
	fi
	done
	if [ "${#secret_files[@]}" -gt 0 ]; then
	exit 1
	fi
	displayName: "azdev scan - Pro on Modified Extensions"
	- job: AzdevScanModifiedExtensionsMedium
	displayName: "azdev scan - Medium Priority on Modified Extensions"
	dependsOn: AzdevScanModifiedExtensionsHigh
	condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
	continueOnError: true
	pool:
	name: 'pool-ubuntu-2004'
	steps:
	- task: UsePythonVersion@0
	displayName: 'Use Python 3.11'
	inputs:
	versionSpec: 3.11
	- template: .azure-pipelines/templates/azdev_setup.yml
	- bash: |
	#!/usr/bin/env bash
	set -ev
	source ./env/bin/activate
	git fetch origin --depth=1 $(System.PullRequest.TargetBranch)
	declare -A secret_files
	for FILE in `git diff --name-only --diff-filter=AM origin/$(System.PullRequest.TargetBranch)` ; do
	detected=$(azdev scan --confidence-level MEDIUM -f $FILE --continue-on-failure| python -c "import sys, json; print(json.load(sys.stdin)['secrets_detected'])")
	if [ $detected == 'True' ]; then
	printf "\033[0;31mSecrets detected from %s, Please remove or replace it. You can run 'azdev scan'/'azdev mask' locally to fix.\033[0m" "$FILE"
	secret_files+=$FILE
	fi
	done
	if [ "${#secret_files[@]}" -gt 0 ]; then
	exit 1
	fi
	displayName: "Run azdev scan - Medium Priority"

[wangzelin007]

- job: AzdevScanModifiedExtensionsHigh
  displayName: "azdev scan - High Priority on Modified Extensions"
  condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest'))
  pool:
    name: 'pool-ubuntu-2004'
  steps:
    - task: UsePythonVersion@0
      displayName: 'Use Python 3.11'
      inputs:
        versionSpec: 3.11
    - template: .azure-pipelines/templates/azdev_setup.yml
    - bash: |
        #!/usr/bin/env bash
        set -ev
        source ./env/bin/activate
        git fetch origin --depth=1 $(System.PullRequest.TargetBranch)
        declare -A secret_files
        for FILE in `git diff --name-only --diff-filter=AM origin/$(System.PullRequest.TargetBranch)` ; do
          detected=$(azdev scan -f $FILE --continue-on-failure| python -c "import sys, json; print(json.load(sys.stdin)['secrets_detected'])")
          if [ $detected == 'True' ]; then
            printf "\033[0;31mSecrets detected from %s, Please remove or replace it. You can run 'azdev scan'/'azdev mask' locally to fix.\033[0m" "$FILE"
            secret_files+=$FILE
          fi
        done
        if [ "${#secret_files[@]}" -gt 0 ]; then
          exit 1
        fi
      displayName: "Run azdev scan - High Priority"

[wangzelin007]

How about renaming them to something more descriptive, like AzdevScanModifiedExtensionsHigh for the first job and AzdevScanModifiedExtensionsMedium for the second?
Similarly, the displayName fields could be updated to make it easier to differentiate these steps in the logs or UI.

[evelyn-ys]

make sense!