Possible support of Apple Accelerate Framework?

[Ionizing]

Hi,
First of all, thank you for your outstanding work on the rstsr crate — it’s shaping up to be one of the most promising foundational libraries for scientific computing in Rust.

While exploring the repository, I noticed that rstsr already supports multiple BLAS/LAPACK backends such as MKL, AOCL, and KML, which is impressive. I primarily work on macOS, so I was wondering whether it would be possible (or practical) for rstsr to utilize the Apple Accelerate Framework for optimized BLAS/LAPACK performance.

I understand this could be a non-trivial task and that maintainers may not have time to implement it. I’m interested in experimenting with this myself — ideally with your guidance if possible.

I’d appreciate it if you could help clarify a few points before I get started:

1. What’s the recommended way to create a new FFI crate for a backend?
2. Roughly how many BLAS/LAPACK routines does a backend need to cover for rstsr compatibility?
3. Are threading-related routines relevant for integration?
4. What kind of tests should the backend pass to ensure correctness?

Additionally, I noticed that OpenBLAS’s cblas.h differs from Netlib’s reference version. For example, OpenBLAS adds:

• cblas_??matcopy: In-place/out-of-place matrix scaling with optional transposition
• cblas_?geadd: General matrix addition (including transposed forms)
• cblas_?gemm_batch: Batched matrix multiplication
• BIGFLOAT and INT8 extensions
• Multithreading interfaces, etc.

Does rstsr currently rely on any of these extended routines? If so, should the Accelerate backend aim to support them as well?

To be clear, this is a personal exploration driven purely by interest, and I don’t expect to complete it fully. Even so, I’d really appreciate your insights and any advice you can share.

Best regards,
Ionizing

[Ionizing]

From what I can tell, macOS includes blas/cblas and lapack/clapack in vecLib, but it doesn’t ship lapacke. So I’m still thinking about whether adding Accelerate support to rstsr makes sense given this gap ......

[ajz34]

comment in simplified Chinese

您好呀！

也感谢您对 RSTSR 项目的支持！我也看到您在 RSTSR 主程序中的 PR 了 ( RESTGroup/rstsr#68 )，看起来也很不错。我有可能会找合适的时候合并 PR，取决于我们是否有 macOS 设备、或持续集成调试与测试手段。

不好意思之前我这边邮件的推送设置好像也出了问题，很长时间没有注意到您的 PR 与 issue，因此现在才作回复。前段时间也在做别的课题，下个月开始可能会重新积极维护 RSTSR 的部分。

实际上同时支持 MKL, AOCL 与 KML 也是因为我们课题组里有这些计算资源 (Intel x86, AMD x86, HiSilicon arm64)；后续可能着重于 RSTSR 的功能完善与类 CUDA 设备的支持。
macOS 确实不是我们优先开发的目标。我想 OpenBLAS 在 macOS 上应该也是支持的，凑活用的话不太清楚是不是可以。不过支持 macOS 最好的方法肯定还是引入 Accelerate，我也有意尽可能扩大所能支持的操作系统与设备的范围。

---

1. What’s the recommended way to create a new FFI crate for a backend?

FFI crate 要做哪些事情，实际上现在还没有稳定下来。所以这个问题我自己还没完全想清楚。我也打算未来写一个具体的开发者指南以厘清思路。

• 从功能导向设计的角度上来说，不同 BLAS distribution 的 FFI crate 需要包含其特有的函数。具体来说，不同的 BLAS distribution 有不同的 BLAS extension 函数、有不同的线程调度函数。这些函数现在或未来有可能要被引入到 RSTSR 的 device-dependent 实现中。
• BLAS/LAPACK 的功能是每个 BLAS distribution 都有、且函数接口完全等同的。这些通常是用软链接的方式，链接 rstsr-lapack-ffi crate 中的 dyload_*.rs 与 ffi_*.rs 以避免文件复制，再手动编辑 mod.rs 文件。
• 对于其他 BLAS distribution 特有的函数，我们需要写一个 script 来自动生成 FFI 代码。

2. Roughly how many BLAS/LAPACK routines does a backend need to cover for rstsr compatibility?

BLAS 会用到不少 (至少 level-3 BLAS 基本上都会用到)，但 LAPACK 我们可能只涵盖一部分。NumPy/SciPy 也有类似的 FFI 层设计，我们对 BLAS/LAPACK 的覆盖程度可能也会参考它们的设计。
在 RSTSR 中，我们一般只使用 BLAS/LAPACK 函数，大多数时候不使用 CBLAS/LAPACKE (未来不一定会支持 LAPACKE 接口的维护，而是在 RSTSR 项目里使用 Rust 编写必要的、等同于 LAPACKE 功能的封装函数)。
这些 BLAS/LAPACK 函数有 rstsr-lapack-ffi crate 已经定义的函数打底了。

3. Are threading-related routines relevant for integration?

是的。某种程度上这也是我开发与 ndarray ( https://github.com/rust-ndarray/ndarray/ ) 不同的张量库的原动力之一。为了编写程序方便，我们很有可能会使用多线程 BLAS；但并行地调用 BLAS 函数容易造成线程超额使用，拖累程序速度。如果集群队列系统比较老旧，也容易影响集群其他用户的使用。线程函数的支持是首先需要被引入的支持功能。

4. What kind of tests should the backend pass to ensure correctness?

如果只是在 FFI 库，我想就不用引入测试了，只要我们手写的 scripts 结合 bindgen 能 work 就行。如果是要在 RSTSR 中引入新的设备支持，测试这件事受限于精力，还没认真搞起来。您已经在 RSTSR 那边提过 PR 了 ( RESTGroup/rstsr#68 )，目前这些简单的测试就是全部了；但肯定这些还不够。

I noticed that OpenBLAS’s cblas.h differs from Netlib’s reference version. Does rstsr currently rely on any of these extended routines? If so, should the Accelerate backend aim to support them as well?

RSTSR 目前还没有引入 BLAS extension 函数；但未来会考虑引入。好处是可以利用一些 BLAS distribution 的特有优化；坏处是每个 BLAS distribution 都要搞自己的特有封装。
OpenBLAS 的 cblas.h 与 Reference LAPACK 的不同，它重新设计了 CBLAS FFI 接口。OpenBLAS 对 cblas.h 的改动比较大 (而且 CBLAS 内容也不算很多)，所以 FFI 那边一整个都重新来了一遍。对应到 KML (Kunpeng Math Library)，它的 kblas 部分其实也是魔改过后的 cblas。

Hello there!

Thank you for your support for the RSTSR project! I also noticed your PR in the main RSTSR program (RESTGroup/rstsr#68), which looks very good. I will likely merge the PR when appropriate, depending on whether we have access to macOS devices or the means for continuous integration debugging and testing.

Sorry, it seems there was an issue with my email notification settings previously, causing me to overlook your PR and issue for a long time. That's why my reply is only coming now. I've also been working on other projects recently, but starting next month, I may become actively involved in maintaining parts of RSTSR again.

In fact, the simultaneous support for MKL, AOCL, and KML is because our research group has access to these computing resources (Intel x86, AMD x86, HiSilicon arm64); subsequent efforts might focus on improving RSTSR's functionality and adding support for CUDA-like devices.
macOS is indeed not our primary development target. I think OpenBLAS should be supported on macOS, though I'm not sure if it would work as a temporary solution. However, the best way to support macOS would definitely be to introduce Accelerate, and I do intend to expand the range of supported operating systems and devices as much as possible.

---

1. What’s the recommended way to create a new FFI crate for a backend?

What exactly an FFI crate needs to accomplish hasn't been stabilized yet. So, I haven't fully figured out this question myself. I also plan to write a specific developer guide in the future to clarify my thoughts.

• From a function-oriented design perspective, FFI crates for different BLAS distributions need to include their specific functions. Specifically, different BLAS distributions have different BLAS extension functions and different thread scheduling functions. These functions might now or in the future be introduced into RSTSR's device-dependent implementations.
• The functionality of BLAS/LAPACK is common to every BLAS distribution, with completely identical function interfaces. These are usually linked using soft links to the dyload_*.rs and ffi_*.rs files in the rstsr-lapack-ffi crate to avoid file duplication, and then the mod.rs file is manually edited.
• For functions specific to other BLAS distributions, we need to write a script to automatically generate the FFI code.

2. Roughly how many BLAS/LAPACK routines does a backend need to cover for rstsr compatibility?

Quite a few BLAS routines are used (at least most of level-3 BLAS), but we might only cover a portion of LAPACK. NumPy/SciPy have a similar FFI layer design, and our coverage of BLAS/LAPACK might refer to their designs.
In RSTSR, we generally only use BLAS/LAPACK functions, and most of the time we don't use CBLAS/LAPACKE (we might not maintain support for the LAPACKE interface in the future; instead, we'll write necessary wrapper functions in Rust within the RSTSR project that are equivalent to LAPACKE functionality).
These BLAS/LAPACK functions are already covered by the functions defined in the rstsr-lapack-ffi crate.

3. Are threading-related routines relevant for integration?

Yes. To some extent, this is also one of the original motivations for me to develop a tensor library different from ndarray (https://github.com/rust-ndarray/ndarray/). For programming convenience, we are very likely to use multi-threaded BLAS; however, calling BLAS functions in parallel can easily cause thread oversubscription, slowing down the program. If the cluster queue system is relatively old, it can also easily affect other users of the cluster. Support for threading functions is a supporting feature that needs to be introduced first.

4. What kind of tests should the backend pass to ensure correctness?

If it's just for the FFI library, I don't think introducing tests is necessary, as long as our hand-written scripts combined with bindgen work. If it's about introducing new device support in RSTSR, testing hasn't been seriously established due to limited energy. You've already submitted a PR to RSTSR (RESTGroup/rstsr#68). Currently, those simple tests are all there is; but they are certainly not enough.

I noticed that OpenBLAS’s cblas.h differs from Netlib’s reference version. Does rstsr currently rely on any of these extended routines? If so, should the Accelerate backend aim to support them as well?

RSTSR currently does not introduce BLAS extension functions; but it will consider introducing them in the future. The advantage is being able to utilize specific optimizations of certain BLAS distributions; the disadvantage is having to create specific wrappers for each BLAS distribution.
OpenBLAS's cblas.h differs from Reference LAPACK; it has redesigned the CBLAS FFI interface. OpenBLAS has made relatively large changes to cblas.h (and CBLAS isn't that extensive anyway), so the entire FFI part had to be redone. Correspondingly, for KML (Kunpeng Math Library), its kblas part is also a modified version of cblas.