llama-cpp: allow to override ROCM GPU targets

[kurnevsky]

It allows to significantly reduce build time in case of building for a single GPU. The same was done for whisper-cpp and llama-cpp flake.

Things done

• Built on platform:
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• Tested, as applicable:
  • NixOS tests in nixos/tests.
  • Package tests at passthru.tests.
  • Tests in lib/tests or pkgs/test for functions and "core" functionality.
• Ran nixpkgs-review on this PR. See nixpkgs-review usage.
• Tested basic functionality of all binary files, usually in ./result/bin/.
• Nixpkgs Release Notes
  • Package update: when the change is major or breaking.
• NixOS Release Notes
  • Module addition: when adding a new NixOS module.
  • Module update: when the change is significant.
• Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

---

Add a 👍 reaction to pull requests you find important.

[kurnevsky]

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 435263
Commit: f9696ce33be44dfd70ec11293d0cdab439409ddd

---

x86_64-linux

✅ 1 package built:

• llama-cpp-rocm

[philiptaron]

Requesting review from known AMD GPU person @K900. Please forward if you know someone better.

[K900]

I know nothing about ROCm, you want @LunNova or @Flakebi probably.

[philiptaron]

I will merge if they approve!

[LunNova]

Would it be better to have a single gpuTargets arg that works for CUDA or HIP depending on what's enabled? That's how some other packages like torch do this.

Otherwise seems fine.

[philiptaron]

I'd rather not invent an abstraction mechanism like that, but if one already exists and is solid, let's consider using it.

[LunNova]

torch:

nixpkgs/pkgs/development/python-modules/torch/source/default.nix

Lines 177 to 193 in 1236127

	gpuTargetString = strings.concatStringsSep ";" (
	if gpuTargets != [ ] then
	# If gpuTargets is specified, it always takes priority.
	gpuTargets
	else if cudaSupport then
	gpuArchWarner supportedCudaCapabilities unsupportedCudaCapabilities
	else if rocmSupport then
	# Remove RDNA1 gfx101x archs from default ROCm support list to avoid
	# use of undeclared identifier 'CK_BUFFER_RESOURCE_3RD_DWORD'
	# TODO: Retest after ROCm 6.4 or torch 2.8
	lib.lists.subtractLists [
	"gfx1010"
	"gfx1012"
	] (rocmPackages.clr.localGpuTargets or rocmPackages.clr.gpuTargets)
	else
	throw "No GPU targets specified"
	);

whisper:

nixpkgs/pkgs/by-name/wh/whisper-cpp/package.nix

Line 25 in 16e0775

rocmGpuTargets ? builtins.concatStringsSep ";" rocmPackages.clr.gpuTargets,

So we already have fragmentation. No strong feelings.

[kurnevsky]

llama-cpp can be built with both ROCM and CUDA enabled at the same time, so it's better not to mix architectures in the same arg.

[kurnevsky]

@philiptaron should this be merged then? :)

[philiptaron]

I'm looking for the approval ☑️