sparta fails to concretize on tuolumne

[rfhaque]

Describe the bug
spack gives the following concretization error for sparta on tuolumne:

     1. sparta-snl: 'amdgpu_target=none' conflicts with '+rocm'
     2. kokkos: 'amdgpu_target=none' conflicts with '+rocm'
     3. sparta-snl: 'amdgpu_target=none' conflicts with '+rocm'
        required because conflict constraint +rocm 
          required because sparta-snl@master+apu+kokkos+mpi fft=fftw3 fft_kokkos=hipfft requested explicitly 
        required because conflict is triggered when amdgpu_target=none 
          required because sparta-snl@master+apu+kokkos+mpi fft=fftw3 fft_kokkos=hipfft requested explicitly 
     4. sparta-snl: 'amdgpu_target=none' conflicts with '+rocm'
        required because conflict is triggered when amdgpu_target=none 
          required because sparta-snl@master+apu+kokkos+mpi fft=fftw3 fft_kokkos=hipfft requested explicitly 
        required because conflict constraint +rocm 
          required because sparta-snl@master+apu+kokkos+mpi fft=fftw3 fft_kokkos=hipfft requested explicitly 
     5. kokkos: 'amdgpu_target=none' conflicts with '+rocm'
        required because conflict is triggered when amdgpu_target=none 
          required because sparta-snl depends on kokkos+rocm when +rocm 
            required because sparta-snl@master+apu+kokkos+mpi fft=fftw3 fft_kokkos=hipfft requested explicitly 
          required because sparta-snl depends on kokkos@3.1:4+shared when +kokkos 
          required because sparta-snl depends on kokkos+apu when +apu 
        required because conflict constraint +rocm 
          required because sparta-snl depends on kokkos+rocm when +rocm 
            required because sparta-snl@master+apu+kokkos+mpi fft=fftw3 fft_kokkos=hipfft requested explicitly 
          required because sparta-snl depends on kokkos@3.1:4+shared when +kokkos 
          required because sparta-snl depends on kokkos+apu when +apu 

To Reproduce
Steps to reproduce the behavior:

1. In the benchpark repo, git checkout sparta_changes
2. bin/benchpark system init --dest=tuolumne-6.4.3-spx llnl-elcapitan cluster=tuolumne rocm=6.4.3 +gtl compiler=cce lapack=intel-oneapi-mkl blas=intel-oneapi-mkl gpumode=SPX
3. bin/benchpark experiment init tuolumne-6.4.3-spx --dest sparta sparta-snl +rocm~openmp fft_kokkos=hipfft fft=fftw3 +gpu-aware-mpi +other +apu exec_mode=perf
4. bin/benchpark setup tuolumne-6.4.3-spx/sparta sparta

Supercomputer (please complete the following information):

• system: tuolumne
• system parameters: llnl-elcapitan cluster=tuolumne rocm=6.4.3 +gtl compiler=cce lapack=intel-oneapi-mkl blas=intel-oneapi-mkl gpumode=SPX
• experiment: sparta-snl
• experiment parameters: sparta-snl +rocm~openmp fft_kokkos=hipfft fft=fftw3 +gpu-aware-mpi +other +apu exec_mode=perf

[scheibelp]

I can reproduce this on tioga, and it seems to go away right before 206ff18 (#931)

@michaelmckinsey1 I think it might be prudent to revert this temporarily (unless you have time to chase the issue). It seems to me that as of that PR a sigificant quantity of info is getting dropped from the envs that benchpark generates. For example on ad7d87e on tioga, the build works:

bin/benchpark system init --dest=tioga llnl-elcapitan cluster=tioga rocm=6.4.3 +gtl compiler=cce
bin/benchpark experiment init tioga --dest sparta sparta-snl +rocm ~openmp fft_kokkos=hipfft fft=fftw3 +gpu-aware-mpi exec_mode=perf
bin/benchpark setup tioga/sparta/ wkp

(dump the spack.yaml:)

spack:
  concretizer:
    unify: true
  specs:
  - sparta-snl@master +mpi+kokkos fft_kokkos=hipfft fft=fftw3 ~openmp ~cuda +rocm
    amdgpu_target=gfx90a
  include:
  - .../benchpark/wkp/tioga/sparta/workspace/software/spack/sparta_snl/packages.yaml
  - .../benchpark/wkp/tioga/sparta/workspace/software/spack/sparta_snl/spack_includes.yaml

same thing on head of develop omits necessary spec info

spack:
  concretizer:
    unify: true
  specs:
  - 'sparta-snl@master +mpi+kokkos fft_kokkos=hipfft fft=fftw3 '
  include:
  - .../benchpark/wkp/tioga/sparta/workspace/software/spack/sparta_snl/packages.yaml
  - .../benchpark/wkp/tioga/sparta/workspace/software/spack/sparta_snl/spack_includes.yaml

I'm unclear how #931 would omit this (I myself would have to examine the changes more closely to get a notion).

Edit: I think this is on account of #931 (comment)