Skip to content

Conversation

armin976
Copy link

Following my issue at #16564, I've create a pull request with a fix. I've tested this locally as written in the Contributing.md, and it works perfectly. The validateNewFunctionWithConstantArguments:305: failed assertion constantValues must not be nil.' no longer shows up when sending a message on iOS with the XCFramework.

This commit:
Adds a resilient wrapper around MTLFunctionConstantValues that records every constant assignment so they can be replayed whenever the underlying Metal object needs to be recreated. Additionaly, this commit hardens pipeline compilation by rebuilding the constant-values container on demand, reapplying cached constants, and warning before falling back to compiling the kernel without constants instead of hard-failing.

  • Function constant bookkeeping: Introduces ggml_metal_cv with space for 128 cached constants, helper routines to apply and record values, automatic deduplication by index, and safeguards against exceeding the storage limit. Whenever the MTLFunctionConstantValues object is missing, the code now recreates it and replays the stored constants before use.
  • Constant setters: Updates the set_int16, set_int32, and set_bool helpers so they all route through the new recorder, guaranteeing that every assignment is captured for later replay.
  • Pipeline compilation fallback: During pipeline compilation, the library tries to ensure constants are materialized, logs a warning if that fails, and (if the constant container is still unavailable) compiles the function without constants while surfacing a warning rather than aborting with an error. Successful compilations continue to populate the pipeline cache as before.

This:

  • Prevents crashes that previously occurred when Metal discarded the constant-values object mid-run by transparently rebuilding the object and replaying the cached constants.
  • Provides clearer diagnostics (warnings instead of hard errors) when falling back to non-specialized kernels, preserving functionality even in edge environments where function constants cannot be materialized.

@armin976 armin976 requested a review from ggerganov as a code owner October 17, 2025 19:16
@github-actions github-actions bot added ggml changes relating to the ggml tensor library for machine learning Apple Metal https://en.wikipedia.org/wiki/Metal_(API) labels Oct 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Apple Metal https://en.wikipedia.org/wiki/Metal_(API) ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant