[vector] add faiss jni to support faiss in java #6985
Conversation
jerry-024
force-pushed
the
support_faiss_jni
branch
2 times, most recently
from
75a5561 to
cd3101d
Compare
jerry-024
force-pushed
the
support_faiss_jni
branch
2 times, most recently
from
f3456ce to
80c6129
Compare
jerry-024
force-pushed
the
support_faiss_jni
branch
from
80c6129 to
186651a
Compare
There was a problem hiding this comment.
Pull request overview
This pull request adds FAISS (Facebook AI Similarity Search) JNI bindings to enable vector similarity search in Java within the Paimon project. The implementation provides a comprehensive Java API wrapper around the native FAISS C++ library with zero-copy data transfer capabilities using direct ByteBuffers.
Key changes:
- New
paimon-faissmodule with JNI bindings for FAISS library - Native C++ implementation with comprehensive JNI layer for index operations
- Java wrapper classes providing type-safe API for vector indexing and search operations
- Build infrastructure including CMake configuration, shell scripts, and GitHub Actions workflows
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pom.xml | Adds new paimon-faiss profile and build exclusions for native artifacts |
| paimon-faiss/pom.xml | Parent POM for FAISS module |
| paimon-faiss/paimon-faiss-jni/pom.xml | Maven configuration for JNI module with SLF4J and test dependencies |
| paimon-faiss/paimon-faiss-jni/src/main/native/* | CMake build configuration and C++ JNI implementation |
| paimon-faiss/paimon-faiss-jni/src/main/java/org/apache/paimon/faiss/* | Java API wrapper classes for FAISS operations |
| paimon-faiss/paimon-faiss-jni/src/test/java/org/apache/paimon/faiss/IndexTest.java | Comprehensive test suite covering index types, operations, and serialization |
| paimon-faiss/paimon-faiss-jni/scripts/build-native.sh | Build script for compiling native libraries across platforms |
| .github/workflows/* | CI/CD configuration for building and testing FAISS module |
| .gitignore | Exclusions for native build artifacts |
| paimon-faiss/paimon-faiss-jni/NOTICE | Licensing attribution for FAISS library |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| try (Index index = IndexFactory.create(DIMENSION, "IDMap,Flat", MetricType.L2)) { | ||
| ByteBuffer vectorBuffer = createVectorBuffer(NUM_VECTORS, DIMENSION); | ||
| ByteBuffer idBuffer = Index.allocateIdBuffer(NUM_VECTORS); | ||
| idBuffer.asLongBuffer(); |
There was a problem hiding this comment.
The result of idBuffer.asLongBuffer() is not used. This statement has no effect and should be removed.
| idBuffer.asLongBuffer(); |
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
|
|
||
| cmake_minimum_required(VERSION 3.30.1) |
There was a problem hiding this comment.
Requiring exactly CMake version 3.30.1 is overly restrictive. This should use a minimum version requirement (e.g., cmake_minimum_required(VERSION 3.17)) rather than an exact version to allow compatibility with newer CMake versions and avoid forcing users to downgrade.
| cmake_minimum_required(VERSION 3.30.1) | |
| cmake_minimum_required(VERSION 3.17) |
| throw new ExceptionInInitializerError(e); | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Missing JavaDoc for the public getVersion() method. All public API methods should have documentation explaining their purpose and return values.
| /** | |
| * Get the version of the underlying Faiss library. | |
| * | |
| * @return the Faiss library version string | |
| */ |
2 participants
add faiss jni to support faiss in java
Tests
API and Format
Documentation