|
| 1 | +# Faiss SVS Binary Size Comparison |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This document describes the methodology for comparing binary sizes between the default `faiss` target (without SVS) and the `faiss_svs` target (with SVS support). |
| 6 | + |
| 7 | +## Background |
| 8 | + |
| 9 | +SVS (Scalable Vector Search) is an optional feature in Faiss that adds additional index implementations. Including SVS increases binary size, so it was separated into an opt-in target to reduce binary bloat for users who don't need SVS functionality. |
| 10 | + |
| 11 | +## Targets |
| 12 | + |
| 13 | +| Target | Description | |
| 14 | +|--------|-------------| |
| 15 | +| `//faiss:faiss` | Default Faiss library without SVS | |
| 16 | +| `//faiss:faiss_svs` | Faiss library with SVS support | |
| 17 | +| `//faiss:pyfaiss` | Python bindings without SVS | |
| 18 | +| `//faiss:pyfaiss_svs` | Python bindings with SVS | |
| 19 | + |
| 20 | +## Methodology |
| 21 | + |
| 22 | +### Why `demo_sift1M`? |
| 23 | + |
| 24 | +The `demo_sift1M` binary is used for comparison because it uses `index_factory`, which creates indexes dynamically from string descriptions at runtime. This means: |
| 25 | + |
| 26 | +- The linker cannot determine at link time which index implementations will be used |
| 27 | +- All index implementations must be included in the binary |
| 28 | +- This provides a worst-case (maximum) binary size measurement |
| 29 | + |
| 30 | +### Build Commands |
| 31 | + |
| 32 | +```bash |
| 33 | +# Build without SVS |
| 34 | +buck2 build @mode/opt fbcode//faiss/demos:demo_sift1M --show-full-output |
| 35 | + |
| 36 | +# Build with SVS |
| 37 | +buck2 build @mode/opt fbcode//faiss/demos:demo_sift1M_svs --show-full-output |
| 38 | +``` |
| 39 | + |
| 40 | +### Size Comparison |
| 41 | + |
| 42 | +```bash |
| 43 | +# Check sizes |
| 44 | +ls -lh <path_to_demo_sift1M> |
| 45 | +ls -lh <path_to_demo_sift1M_svs> |
| 46 | + |
| 47 | +# Verify no SVS symbols in default build |
| 48 | +nm <path_to_demo_sift1M> | grep -i svs |
| 49 | +``` |
| 50 | + |
| 51 | +## Results |
| 52 | + |
| 53 | +| Configuration | Binary Size | |
| 54 | +|---------------|-------------| |
| 55 | +| Without SVS (`faiss`) | 33 MB | |
| 56 | +| With SVS (`faiss_svs`) | 41 MB | |
| 57 | +| **Difference** | **8.24 MB (20% reduction)** | |
| 58 | + |
| 59 | +## Files Changed |
| 60 | + |
| 61 | +The following files were modified to separate SVS from the default Faiss build: |
| 62 | + |
| 63 | +### `faiss/xplat.bzl` |
| 64 | +- Separated SVS source and header files into dedicated functions: |
| 65 | + - `svs_header_files()` - SVS header files |
| 66 | + - `svs_source_files()` - SVS source files |
| 67 | + |
| 68 | +### `faiss/BUCK` |
| 69 | +- Default `faiss` target no longer includes SVS |
| 70 | +- Added new `faiss_svs` target with SVS support |
| 71 | +- Updated `faiss_no_multithreading` and `faiss_omp_mock` to exclude SVS |
| 72 | + |
| 73 | +### `faiss/python/defs.bzl` |
| 74 | +- Added `with_svs` parameter to `pyfaiss_binary()` macro |
| 75 | + |
| 76 | +### `faiss/python/BUCK` |
| 77 | +- Added `pyfaiss_svs` target |
| 78 | + |
| 79 | +### `faiss/fbcode.bzl` |
| 80 | +- Updated `pyfaiss_libraries()` to include SVS variant |
| 81 | + |
| 82 | +### `faiss/tests/BUCK` |
| 83 | +- Updated SVS tests to use `faiss_svs` and `pyfaiss_svs` |
| 84 | + |
| 85 | +### `faiss/demos/BUCK` |
| 86 | +- Added `demo_sift1M_svs` for binary size comparison |
| 87 | + |
| 88 | +## Usage |
| 89 | + |
| 90 | +### For users who need SVS |
| 91 | + |
| 92 | +Replace dependencies on `//faiss:faiss` with `//faiss:faiss_svs`: |
| 93 | + |
| 94 | +```python |
| 95 | +# BUCK file |
| 96 | +cpp_binary( |
| 97 | + name = "my_binary", |
| 98 | + srcs = ["main.cpp"], |
| 99 | + deps = ["//faiss:faiss_svs"], # Use faiss_svs for SVS support |
| 100 | +) |
| 101 | +``` |
| 102 | + |
| 103 | +For Python: |
| 104 | +```python |
| 105 | +python_binary( |
| 106 | + name = "my_script", |
| 107 | + srcs = ["main.py"], |
| 108 | + deps = ["//faiss:pyfaiss_svs"], # Use pyfaiss_svs for SVS support |
| 109 | +) |
| 110 | +``` |
| 111 | + |
| 112 | +### For users who don't need SVS |
| 113 | + |
| 114 | +No changes needed - the default `//faiss:faiss` and `//faiss:pyfaiss` targets now exclude SVS automatically. |
0 commit comments