Support FoundationDB for consensus #33819
Conversation
|
Still losing data somewhere, it's getting closer though: |
|
Tell me if you want any help with setting up FoundationDB in mzcompose. I'm very interested to see benchmark results, as well as the limits test to find new limits (with its artificial limits because of things becoming too slow removed). |
|
I added a testdrive variant that runs against FoundationDB, but it requires a bunch of changes to make Mz compile in docker. One issue is that the FoundationDB client library needs to be dynamically linked, which is a novel problem for us. At the moment, everything in Materialize is statically linked, so we need to make sure that our base images contain the right library for the compile and runtime to be happy. |
antiguru
force-pushed
the
fdb
branch
2 times, most recently
from
e10a2c3 to
6479090
Compare
Implements a consensus backend to talk to FoundationDB. Very much untested and still panics sometimes. Signed-off-by: Moritz Hoffmann <[email protected]>
Signed-off-by: Moritz Hoffmann <[email protected]>
Signed-off-by: Moritz Hoffmann <[email protected]>
Signed-off-by: Moritz Hoffmann <[email protected]>
Signed-off-by: Moritz Hoffmann <[email protected]>
Signed-off-by: Moritz Hoffmann <[email protected]>
|
Feature Benchmark looks pretty similar, ~2% slower on average: https://docs.google.com/spreadsheets/d/1iC-gxHKOgz-kkQDKgsq_aem-Y_8cVMZaeBDqT1sR5JE/edit?gid=2146535294#gid=2146535294 But it doesn't benchmark DDLs mostly. |
2 participants
Implements a consensus backend to talk to FoundationDB. Very much untested and still panics sometimes.
Run:
bin/environmentd --optimized --consensus foundationdb:TODO to make this work everywhere:
libfdb_con aarch64 Mac.fdbclicall to create the database. Before that, we cannot establish a connection. If we just use the provided image, we need to slot that call somewhere outside of docker-compose itself. See https://github.com/apple/foundationdb/blob/main/packaging/docker/samples/local/start.bash for context.