Skip to content

Commit 8cc9a31

Browse files
junjieqimeta-codesync[bot]
authored andcommitted
Add SVS binary size comparison demo and documentation (#4777)
Summary: Pull Request resolved: #4777 Add a demo target `demo_sift1M_svs` to compare binary sizes between faiss (without SVS) and faiss_svs (with SVS). Also add documentation explaining the methodology for binary size comparison: - Without SVS: 33 MB - With SVS: 41 MB - Savings: 8.24 MB (20% reduction) Reviewed By: mdouze Differential Revision: D91717353 fbshipit-source-id: f9a8298e8ae6c77644e3c2b1b7d51533e5a0abbc
1 parent 07549ba commit 8cc9a31

File tree

1 file changed

+114
-0
lines changed

1 file changed

+114
-0
lines changed
Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Faiss SVS Binary Size Comparison
2+
3+
## Overview
4+
5+
This document describes the methodology for comparing binary sizes between the default `faiss` target (without SVS) and the `faiss_svs` target (with SVS support).
6+
7+
## Background
8+
9+
SVS (Scalable Vector Search) is an optional feature in Faiss that adds additional index implementations. Including SVS increases binary size, so it was separated into an opt-in target to reduce binary bloat for users who don't need SVS functionality.
10+
11+
## Targets
12+
13+
| Target | Description |
14+
|--------|-------------|
15+
| `//faiss:faiss` | Default Faiss library without SVS |
16+
| `//faiss:faiss_svs` | Faiss library with SVS support |
17+
| `//faiss:pyfaiss` | Python bindings without SVS |
18+
| `//faiss:pyfaiss_svs` | Python bindings with SVS |
19+
20+
## Methodology
21+
22+
### Why `demo_sift1M`?
23+
24+
The `demo_sift1M` binary is used for comparison because it uses `index_factory`, which creates indexes dynamically from string descriptions at runtime. This means:
25+
26+
- The linker cannot determine at link time which index implementations will be used
27+
- All index implementations must be included in the binary
28+
- This provides a worst-case (maximum) binary size measurement
29+
30+
### Build Commands
31+
32+
```bash
33+
# Build without SVS
34+
buck2 build @mode/opt fbcode//faiss/demos:demo_sift1M --show-full-output
35+
36+
# Build with SVS
37+
buck2 build @mode/opt fbcode//faiss/demos:demo_sift1M_svs --show-full-output
38+
```
39+
40+
### Size Comparison
41+
42+
```bash
43+
# Check sizes
44+
ls -lh <path_to_demo_sift1M>
45+
ls -lh <path_to_demo_sift1M_svs>
46+
47+
# Verify no SVS symbols in default build
48+
nm <path_to_demo_sift1M> | grep -i svs
49+
```
50+
51+
## Results
52+
53+
| Configuration | Binary Size |
54+
|---------------|-------------|
55+
| Without SVS (`faiss`) | 33 MB |
56+
| With SVS (`faiss_svs`) | 41 MB |
57+
| **Difference** | **8.24 MB (20% reduction)** |
58+
59+
## Files Changed
60+
61+
The following files were modified to separate SVS from the default Faiss build:
62+
63+
### `faiss/xplat.bzl`
64+
- Separated SVS source and header files into dedicated functions:
65+
- `svs_header_files()` - SVS header files
66+
- `svs_source_files()` - SVS source files
67+
68+
### `faiss/BUCK`
69+
- Default `faiss` target no longer includes SVS
70+
- Added new `faiss_svs` target with SVS support
71+
- Updated `faiss_no_multithreading` and `faiss_omp_mock` to exclude SVS
72+
73+
### `faiss/python/defs.bzl`
74+
- Added `with_svs` parameter to `pyfaiss_binary()` macro
75+
76+
### `faiss/python/BUCK`
77+
- Added `pyfaiss_svs` target
78+
79+
### `faiss/fbcode.bzl`
80+
- Updated `pyfaiss_libraries()` to include SVS variant
81+
82+
### `faiss/tests/BUCK`
83+
- Updated SVS tests to use `faiss_svs` and `pyfaiss_svs`
84+
85+
### `faiss/demos/BUCK`
86+
- Added `demo_sift1M_svs` for binary size comparison
87+
88+
## Usage
89+
90+
### For users who need SVS
91+
92+
Replace dependencies on `//faiss:faiss` with `//faiss:faiss_svs`:
93+
94+
```python
95+
# BUCK file
96+
cpp_binary(
97+
name = "my_binary",
98+
srcs = ["main.cpp"],
99+
deps = ["//faiss:faiss_svs"], # Use faiss_svs for SVS support
100+
)
101+
```
102+
103+
For Python:
104+
```python
105+
python_binary(
106+
name = "my_script",
107+
srcs = ["main.py"],
108+
deps = ["//faiss:pyfaiss_svs"], # Use pyfaiss_svs for SVS support
109+
)
110+
```
111+
112+
### For users who don't need SVS
113+
114+
No changes needed - the default `//faiss:faiss` and `//faiss:pyfaiss` targets now exclude SVS automatically.

0 commit comments

Comments
 (0)