Latest azure-cli has compatible issues with azureml packages

[huliang-microsoft]

Describe the bug

We have an ADO pipeline installing these packages:
azureml
azureml-core
azure-cli
azure-ai-ml
azure-identity
azureml-mlflow
...
it started to fail with weird error like this 5-6 hours ago:

      running build_ext
      building '_ruamel_yaml' extension
      creating build/temp.linux-x86_64-3.9
      creating build/temp.linux-x86_64-3.9/ext
      x86_64-pc-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fcommon -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fcommon -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fcommon -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -fPIC -I/usr/include/python3.9 -c ext/_ruamel_yaml.c -o build/temp.linux-x86_64-3.9/ext/_ruamel_yaml.o
      ext/_ruamel_yaml.c:4:10: fatal error: Python.h: No such file or directory
          4 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-pc-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> ruamel.yaml

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

After trying to install package separately, I found that it is azure-cli and azureml packages and after reverting the azure-cli to 2.69.0, it was all good.

Related command

Not Az cli command issue.

Errors

  running build_ext
  building '_ruamel_yaml' extension
  creating build/temp.linux-x86_64-3.9
  creating build/temp.linux-x86_64-3.9/ext
  x86_64-pc-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fcommon -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fcommon -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -fcommon -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -DOPENSSL_NO_SSL3 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_COMP -fPIC -I/usr/include/python3.9 -c ext/_ruamel_yaml.c -o build/temp.linux-x86_64-3.9/ext/_ruamel_yaml.o
  ext/_ruamel_yaml.c:4:10: fatal error: Python.h: No such file or directory
      4 | #include "Python.h"
        |          ^~~~~~~~~~
  compilation terminated.
  error: command '/usr/bin/x86_64-pc-linux-gnu-gcc' failed with exit code 1
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> ruamel.yaml

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Issue script & Debug output

Same as Error

Expected behavior

Packages should be installed compatible.

Environment Summary

azure-cli 2.70.0

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.