Priority setting for liboqs 0.16.0

[dstebila]

With the imminent release of liboqs 0.15.0, it's time to turn our attention to features and updates people would like to make for 0.16.0.

Feel free to use this discussion board to brainstorm things you'd like to contribute and discuss.

The liboqs contribution wishlist has a high-level view of some desirable features identified in the past few months. There's also a start of a 0.16.0 milestone and a plethora of open issues.

@baentsch @xuganyu96

[xuganyu96]

SLH-DSA C integration refactor

I have two proposals for improving SLH-DSA C integration:

• Bring SLH-DSA's integration in line with other algorithms: SLH-DSA metadata is not stored in copy_from_upstream.yml. SLH-DSA integration uses a separate script copy_from_slh_dsa_c.py instead of copy_from_upstream.py. SLH-DSA's documentation under docs/ is generated from a template, unlike all other hard-coded documentations.
• Trim the number of pre-hashed variants: there are 144 pre-hashed variants of SLH-DSA (12 hash functions times 12 parameter sets). It is infeasible to manage them by hand, and automated tools consume substantial developer time. I want to explore implementing an alternative API for pre-hashed variants that are more in line with FIPS 205's Algorithm 23 (see snippet below from slhdsa-c) with "pre-hash" passed in as an argument ph.

size_t hash_slh_sign(uint8_t *sig, const uint8_t *m, size_t m_sz,
                     const uint8_t *ctx, size_t ctx_sz, const char *ph,
                     const uint8_t *sk, const uint8_t *addrnd,
                     const slh_param_t *prm);

These two points intertwine with each other, and there are implications for downstream projects. I will need to try a few PoC's before coming up with an implementation strategy.

[baentsch]

Thanks for the assessment and proposals @xuganyu96 ! fwiw, trimming the number of pre-hashed variants seems very sensible. Two suggestions for that: 1) Do we know anyone using SLH-DSA via liboqs and could ask which pre-hashed variants they need and focus on those? 2) Look at what other libraries offer, e.g., OpenSSL, and mirror that?

The latter suggestion of course triggers the question whether it is worth the effort to "polish" SLH-DSA in liboqs to begin with: It so far did not get integrated to oqs-provider (probably because OpenSSL already supports it) and thus is quite unlikely to ever land there -- begging the question why this is done at all? Who should consume it in which way? Also, SLH-DSA does not fit the bill of "experimental PQ algorithm" any more and any work on it seems like a distraction to the OQS goals, no? Again the question: Are we aware of anyone using SLH-DSA via liboqs?

Final question: Is the code owner of this (@h2parson if I'm not mistaken?) interested in helping to eliminate the shortcomings identified above? Also, if this is a PQCA-origin code base (?) wouldn't it make sense to ask its authors whether they'd be willing to provide a more "aligned" integration (making available META.ymls, such as for OQS to use the copy_from_upstream logic unchanged -- much along the way done for PQCP's MLKEM)?

[xuganyu96]

Do we know anyone using SLH-DSA via liboqs and could ask which pre-hashed variants they need and focus on those?

This is not quite what I meant with trimming SLH-DSA pre-hashed variants.
I was not proposing "choosing some of the 144 variants." Instead, I was proposing to change the signing function signature so that choosing different pre-hash algorithms is done by passing different values to an argument instead of calling different functions.
For example, instead of calling:

• OQS_SIG_slh_dsa_sha2_224_prehash_sha2_128s_sign
• OQS_SIG_slh_dsa_sha2_256_prehash_sha2_128s_sign
• OQS_SIG_slh_dsa_sha2_384_prehash_sha2_128s_sign

The caller would call:

• OQS_SIG_slh_dsa_prehash_sha2_128s_sign(..., "sha2_224")
• OQS_SIG_slh_dsa_prehash_sha2_128s_sign(..., "sha2_256")
• OQS_SIG_slh_dsa_prehash_sha2_128s_sign(..., "sha2_384")

The number of function signatures for pre-hashed variants should reduce from 144 to 12, but all 144 possible combinations of pre-hash algorithms and parameters would still all be accessible.
The latter API already exists in the upstream implementation so no additional wrapping is needed.

Look at what other libraries offer, e.g., OpenSSL, and mirror that?

OpenSSL only offers the "pure" variants, not the pre-hashed variants.

whether it is worth the effort to "polish" SLH-DSA in liboqs to begin with

A big part of my concern with SLH-DSA is actually not with the algorithm itself, but with the status quo of the infrastructure used to integrate it into liboqs, especially how SLH-DSA alone has bespoke scaffolding that is separate from everyone else.
See here for an example of how having parallel integration scripts makes future maintenance work more difficult.

Final question: Is the code owner of this (@h2parson if I'm not mistaken?) interested in helping to eliminate the shortcomings identified above?

Hayden is (at this moment) no longer with OQS. I will be in charge of carrying out the refactor.

[baentsch]

I was proposing to change the signing function signature

This then is an ABI and API breaking change impacting all users of OQS. I'm not sure this the best way forward, particularly considering it serves to support one single algorithm only.

The OQS premise (promise, actually) was that it delivers one stable API behind which all PQC algs can be utilized, regardless of how many algs are supported and what "peculiarities" each one may have.

Changing an API requires us to change all integrations and requires all users to change their code. Such change imo should be done only if we can demonstrate clear additional value for users. Can we do that? Are there users itching for this feature? Do we know whether users will be willing to adapt their code even though they don't need SLH-DSA? By way of example: No-one seems to want this in the "OpenSSL ecosystem" as otherwise SLH-DSA would have been added to oqsprovider or pre-hashed variants would have been added to OpenSSL default provider, which...

OpenSSL only offers the "pure" variants, not the pre-hashed variants.

... also doesn't seem to be the case/a requirement.

The latter API already exists in the upstream implementation so no additional wrapping is needed.

This argument speaks to the change being simple to implement in liboqs, but doesn't speak to the impact this has on the library users. I see it as a library maintainer's obligation to also consider the impact API decisions have on library users -- as arguably there are (many) more using this code than there are people developing it.

See here for an example of how having parallel integration scripts makes future maintenance work more difficult.

I do recall this comment and (as I made it :) I completely agree with the conclusion that the SLH-DSA integration is hardly maintainable. Had I been able to review the PR landing it, I'd have demanded changes for the very reasons that it's creating a "second class" algorithm in the collection, making life hard for anyone tasked with maintaining it and also to mitigate the risk of the code owner leaving the project ...

Hayden is (at this moment) no longer with OQS

...which has materialized.

If we indeed want to invest time in another standardized algorithm --which I personally find wrong as it detracts us both from working on adding new algorithms to it as well as making the library better for all algorithms and users-- my counter-proposal would be the addition of a new signature API instead: That way no current liboqs users will be impacted. The new API should still serve all algorithms such that liboqs would be able to deprecate the current API in a possible (imo unlikely) future where everyone has switched to the new API. This will create more work in liboqs (new tests, new documentation, a new API to maintain and support, etc.) but is reducing the work imposed on OQS users.

[xuganyu96]

It feels like there is still some discrepancy between our understandings on the exact changes I am proposing for pre-hashed SLH-DSA's API and the values they bring (or so I hope they bring). At this point I think it's more effective to discuss this synchronously (on the next status call?), where we can also get other stakeholders to chime in.

On the other hand, I think we have reached consensus that the integration scripts could use some refactor, and I can start working on that right away.

[xuganyu96]

Note: NIST defined 12 OID's for the pre-hash variants of SLH-DSA instead of 144.

[baentsch]

I like the 0.16.0 milestone list -- with the comment that an ML-DSA update does not seem as urgent as an HQC update: The latter has been requested, the former seems primarily a courtesy to a "sibling PQCA" project (and does not align with OQS' focus on new, experimental --read: non standardized-- algorithms). I'd thus suggest prioritizing #2192 and #2118 (adding them to the milestone) and dropping #2272 (accepting a PR if it comes, of course). Also, I wonder what could be done to add more on-ramp sig algs? I'm sure @dstebila has (had?) contacts with those authors: Any statement of interest to help us add their work to liboqs?

[dstebila]

I'd thus suggest prioritizing #2192 and #2118 (adding them to the milestone)

Done.

and dropping #2272 (accepting a PR if it comes, of course).

I'm inclined to keep it in for now so that it's clear it's something we would be very happy to have. But we don't have to block the release on it, if and when it comes to it.

Also, I wonder what could be done to add more on-ramp sig algs? I'm sure @dstebila has (had?) contacts with those authors: Any statement of interest to help us add their work to liboqs?

I can make an effort to pull in some contributions on this. I think the leading candidates to focus on are SQIsign, FAEST, and SDith, since we have had expressions of interest already in them. LESS and HAWK also have authors that interacted with OQS before. What do you think?

[RodriM11]

I believe it would be ideal to have a pool of candidates pertaining to each of the different families currently on play in the on-ramp process (focusing as much as possible in the leading candidates, as @dstebila mentions). I agree SQIsign, FAEST, SDitH, HAWK and LESS are some of the leading candidates (along with MAYO, CROSS and UOV, already present), but I believe it would be great for at least one MPCith (Mirath, MQOM, PERK or RYDE) to also be present, since I imagine at least one will advance to subsequent rounds.

[baentsch]

"pull" -> "poll" or "pool"? And: Do you have contact(s) to any of these alg's authors such as to ask?

[dstebila]

Rodrigo, the list of signature on-ramp algorithms in the wishlist was organized based on where we'd received expressions of interest. I think you make a good point about also trying to cover different families, and getting an MPC-in-the-head scheme in sounds like a good idea indeed.

[RodriM11]

"pull" -> "poll" or "pool"?

Meant "pool" as in variety, sorry!

Do you have contact(s) to any of these alg's authors such as to ask?

Not formally, beyond some known emails of specifications or have coincided on conferences with some authors, but I could try and reach out of the blue in case no other person here has contact with authors of schemes.

[dstebila]

I've created a Github project board for organizing issues and pull request priorities, which may be more helpful than just the milestone.