GPU support for Qdrant

[akshm-weboccult]

I want to use GPU for vector matching. Can we use GPU for vector matching and indexing for faster results.

[generall]

Hi @aksh-wot, according to our benchmarks, there are no significant speedup considering the price of GPU.
The only place where it can do the difference is linear search (at least I didn't see any implementations which would provide meaningful GPU support for HNSW).

We will continue to investigate possibilities from our side, but the results are expected not earlier than second half of 2024

[vmarchenkoff]

Hi! Any updates on this? We noticed enabling GPU support in Dockerfile, does it mean gpu utilization for vector search?

Thank you!

[timvisee]

Almost, for vector indexing - more details later!

[vmarchenkoff]

Thank you for such a quick response, we will follow the announcements!

[timvisee]

@vmarchenkoff More information here! 😄

https://qdrant.tech/blog/qdrant-1.13.x/

[v1ckxy]

Hi, good morning.
Considering that vulkan "works on everything", could you please release a windows binary with vulkan acceleration as well?
And not just a docker for linux, there's people (like me) that doesn't use containers but just run things baremetal.

Regards,

[timvisee]

We don't have binaries for all possible configurations because there's many, but also in the case of GPU, there's a bunch of dependencies we cannot include in the binary itself.

For now I'd recommend to build the binary from scratch. That'll ensure you'll get the best possible optimized binary for your machine.

You can build through cargo (installed with rustup):

git clone https://github.com/qdrant/qdrant.git
cd qdrant
cargo build --features gpu --locked --release
cargo run --features gpu --locked --release