Merge pull request #6 from questdb/corrosion

Rewrite of the core logic in Rust. Float serialisation issues fixed.

Author: amunra

.gitignore

@@ -1,3 +1,5 @@
+target
+Cargo.lock
 build
 .vs
 .vscode

BUILD.md

@@ -5,11 +5,26 @@ This page describes how to build this project.
 Also make sure to read the page on
 [integrating this library into your project](DEPENDENCY.md).
 
-## Pre-requisites
+## Pre-requisites and dependencies
 
-* A modern C11/C++17 compiler.
+* Rust 1.61 or newer (get it from [https://rustup.rs/](https://rustup.rs/))
+* A modern C11 or C++17 compiler.
 * CMake 3.15 or newer.
 
+The library statically links all its dependencies.
+
+```
+$ ls build/libquestdb_client.*
+build/libquestdb_client.a  build/libquestdb_client.so
+$ ldd build/libquestdb_client.so
+        linux-vdso.so.1 (0x00007ffddd344000)
+        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe61d252000)
+        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe61d22f000)
+        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe61d229000)
+        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe61d037000)
+        /lib64/ld-linux-x86-64.so.2 (0x00007fe61d2ee000)
+```
+
 ## Build steps
 
 ### Linux / MacOS
@@ -37,20 +52,9 @@ Visual Studio Code should also work well provided you have the "C/C++",
 
 ## Build outputs
 
-The build will generate libraries compiled to `./build`
+The build will generate both static and dynamic libraries compiled to `./build`
 (or your otherwise selected CMake build directory).
 
-You will find one dynamic library and depending on
-the operating system either one or two static libraries.
-
-On platforms that support compiling with position independent code (Linux, Mac)
-we ship both a static library with `-fPIC` enabled and one with the option
-disabled. Use the former if you intend to link the static library into a dynamic
-library and use the latter if you intend to link it into an executable.
-If you intend to create your own language binding (e.g. for Python), then you
-probably want to use the `-fPIC` static library.
-On Windows there is just one static library you may use for all purposes.
-
 ## Running tests
 
 ### Unit Tests
@@ -87,3 +91,20 @@ Delete the `./build` directory.
 ```bash
 $ rm -fR build  # or your otherwise selected CMake build directory.
 ```
+
+## Building Rust Code
+
+You may find that you need to build the Rust code directly.
+
+From the root:
+
+```
+cargo build --release --features ffi
+```
+
+Note that this will build binaries in `./target` rather than `./build`.
+
+This will also refresh the `line_sender.h` header file.
+
+Call this command if you want to integrate the library within your project and
+you use a build system other than CMake.

CMakeLists.txt

@@ -16,11 +16,30 @@ set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS OFF)
 
-set(LIB_SOURCES
-    src/aborting_malloc.c
-    src/line_sender.c
-    src/mem_writer.c
-    src/utf8.c)
+option(
+    BUILD_SHARED_LIBS
+    "Build shared library dependencies instead of static."
+    OFF)
+
+# Build static and dynamic lib written in Rust by invoking `cargo`.
+# Imports `questdb_client` target.
+add_subdirectory(corrosion)
+corrosion_import_crate(
+    MANIFEST_PATH Cargo.toml
+    FEATURES ffi
+    LINK_AS C)
+target_include_directories(
+    questdb_client INTERFACE
+    ${CMAKE_CURRENT_SOURCE_DIR}/include)
+if(WIN32)
+    set_target_properties(
+        questdb_client-shared
+        PROPERTIES
+        DEFINE_SYMBOL "LINESENDER_DYN_LIB")
+    target_link_libraries(
+        questdb_client-shared
+        INTERFACE wsock32 ws2_32)
+endif(WIN32)
 
 function(set_compile_flags TARGET_NAME)
     if(MSVC)
@@ -43,86 +62,42 @@ function(set_compile_flags TARGET_NAME)
     endif()
 endfunction()
 
-# Shared Library
-add_library(
-    c_questdb_client SHARED
-    ${LIB_SOURCES})
-target_include_directories(
-    c_questdb_client
-    PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
-set_compile_flags(c_questdb_client)
-target_compile_definitions(
-    c_questdb_client
-    PUBLIC
-    LINESENDER_DYN_LIB)
-if(WIN32)
-    set_target_properties(
-        c_questdb_client
-        PROPERTIES
-        DEFINE_SYMBOL "LINESENDER_EXPORTS")
-    target_link_libraries(c_questdb_client wsock32 ws2_32)
-endif(WIN32)
-if(UNIX)
-    target_compile_options(
-        c_questdb_client PRIVATE
-        -fvisibility=hidden)
-endif(UNIX)
-
-# Static Library
-add_library(
-    c_questdb_client_static STATIC
-    ${LIB_SOURCES})
-target_include_directories(
-    c_questdb_client_static
-    PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
-set_compile_flags(c_questdb_client_static)
-if(WIN32)
-    target_link_libraries(c_questdb_client_static wsock32 ws2_32)
-endif(WIN32)
-
-# Static Library w/ -fPIC
-if(UNIX)
-    add_library(
-        c_questdb_client_static_pic STATIC
-        ${LIB_SOURCES})
-    set_property(
-        TARGET c_questdb_client_static_pic
-        PROPERTY POSITION_INDEPENDENT_CODE ON)
-    target_include_directories(
-        c_questdb_client_static_pic
-        PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
-    set_compile_flags(c_questdb_client_static_pic)
-endif(UNIX)
-
 # Examples
 add_executable(
     line_sender_c_example
     examples/line_sender_c_example.c)
 target_link_libraries(
     line_sender_c_example
-    c_questdb_client_static)  # example depending on static lib.
+    questdb_client)
 add_executable(
     line_sender_cpp_example
     examples/line_sender_cpp_example.cpp)
 target_link_libraries(
     line_sender_cpp_example
-    c_questdb_client)  # example depending on dynamic lib.
+    questdb_client)
 
-# Unit test binary.
-add_executable(
+# Unit test binaries.
+function(compile_test TARGET_NAME)
+    list(POP_FRONT ARGV)  # compile_test
+    add_executable(
+        ${TARGET_NAME}
+        ${ARGV})
+    target_link_libraries(
+        ${TARGET_NAME}
+        questdb_client)
+    target_include_directories(
+        ${TARGET_NAME}
+        PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
+    set_compile_flags(${TARGET_NAME})
+    add_test(
+        NAME ${TARGET_NAME}
+        COMMAND ${TARGET_NAME})
+endfunction()
+
+compile_test(
     test_line_sender
     test/mock_server.cpp
     test/test_line_sender.cpp)
-target_link_libraries(
-    test_line_sender
-    c_questdb_client_static)
-target_include_directories(
-    test_line_sender
-    PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/include)
-set_compile_flags(test_line_sender)
-add_test(
-    NAME test_line_sender
-    COMMAND test_line_sender)
 
 # System testing Python3 script.
 # This will download the latest QuestDB instance from Github,

Cargo.toml

@@ -0,0 +1,27 @@
+[package]
+name = "rs-questdb-client"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+name = "questdb_client"
+crate-type = ["cdylib", "staticlib"]
+
+[dependencies]
+libc = "0.2"
+socket2 = "0.4.4"
+dns-lookup = "1.0.8"
+
+[target.'cfg(windows)'.dependencies]
+winapi = "0.3.9"
+
+[build-dependencies]
+cbindgen = { version = "0.24.2", optional = true, default-features = false }
+
+[features]
+# Build C buildings, but don't generate the header file.
+ffi = []
+
+# Auto-generate the header. This is for dev-debugging-diffing only.
+# A hand-crafted header is easier on the eyes.
+gen_h = ["ffi", "cbindgen"]

DEPENDENCY.md

@@ -83,31 +83,25 @@ set(CMAKE_CXX_STANDARD 17)
 set(CMAKE_CXX_STANDARD_REQUIRED ON)
 set(CMAKE_CXX_EXTENSIONS OFF)
 
-find_package(Git REQUIRED)
 include(FetchContent)
 
 FetchContent_Declare(
     c_questdb_client_proj
     GIT_REPOSITORY https://github.com/questdb/c-questdb-client.git
     GIT_TAG CHOSEN_RELEASE_TAG)   # CHANGE ME!
 
-FetchContent_GetProperties(c_questdb_client_proj)
-if(NOT c_questdb_client_proj_POPULATED)
-  FetchContent_Populate(c_questdb_client_proj)
-  add_subdirectory(
-      ${c_questdb_client_proj_SOURCE_DIR}
-      ${c_questdb_client_proj_BINARY_DIR}
-      EXCLUDE_FROM_ALL)
-endif()
+FetchContent_MakeAvailable(c_questdb_client)
 
 add_executable(
     main
     main.cpp)
 target_link_libraries(
     main
-    c_questdb_client_static)
+    questdb_client)
 ```
 
+*Note:* By default, the library will be linked statically. Call `cmake .. -DBUILD_SHARED_LIBS=ON` to depend on the dynamic library.
+
 ## Option 2: CMake add_subdirectory & Git source code grafting
 
 If you're using both CMake and git in your project, then you can graft this
@@ -231,7 +225,7 @@ add_executable(
     main.cpp)
 target_link_libraries(
     main
-    c_questdb_client_static)
+    questdb_client)
 ```
 
 ## Option 3: Other build systems
@@ -251,6 +245,8 @@ If you use a build system other than CMake, the following tips should help you:
   to mark `__declspec(dllexport)`:
   This define should *not* be present when *using* the library.
 
+*Note:* By default, the library will be linked statically. Call `cmake .. -DBUILD_SHARED_LIBS=ON` to depend on the dynamic library.
+
 ## Package Managers
 
 If you are using a particular package manager (e.g. Conan or vcpkg), contact us

DEV_NOTES.md

@@ -0,0 +1,32 @@
+# Developer notes
+
+## CMake Integration
+We temporarily use a forked version of https://github.com/corrosion-rs/corrosion
+to enable linking the Rust crate as C rather than C++.
+
+The "corrosion" directory has been added as:
+
+```
+git subtree add --prefix corrosion https://github.com/amunra/corrosion master --squash
+```
+
+and is being maintained as:
+
+```
+git subtree add --prefix corrosion https://github.com/amunra/corrosion master --squash
+```
+
+Until our outstanding pull request with the upstream project is resolved.
+See: https://github.com/corrosion-rs/corrosion/pull/188.
+
+
+## Building without CMake
+For development, you may also call `cargo build` directly.
+By default, this will not build the `C` FFI layer.
+
+For that, call `cargo build --features ffi`.
+
+If you are editing the C functions and what to see the resulting generated
+header file, call `cargo build --features cbindgen`.
+
+Note that to reduce compile time we don't use cbindgen in the header we ship.

README.md

@@ -6,9 +6,12 @@ This library makes it easy to insert data into [QuestDB](https://questdb.io/).
 This client library implements the [InfluxDB Line Protocol](
 https://questdb.io/docs/reference/api/ilp/overview/) (ILP) over TCP.
 
-* Implementation is in C11, with no dependency on the C++ standard library
-  for simpler inclusion into your projects.
-* The C++ API is a header-only wrapper written in C++17.
+* Implementation is in Rust, with no additional
+  [run-time or link-time dependencies](BUILD.md#pre-requisites-and-dependencies)
+  on the C++ standard library or other libraries.
+* We ship both a static and a dynamic library.
+* The library exposes both a C and a C++17 API.
+* The C++ API is a header-only wrapper over the C API.
 
 ## Protocol
 
@@ -21,7 +24,7 @@ Inserting data into QuestDB can be done via one of three protocols.
 | [PostgreSQL](https://questdb.io/docs/reference/api/postgres/) | Transaction-level | Good |
 
 This library mitigates the lack of confirmation and error reporting by
-validating data ahead of time, before any data is sent to the database instance.
+validating data ahead of time before any data is sent to the database instance.
 
 For example, the client library will report that a supplied string isn't encoded
 in UTF-8. Some issues unfortunately can't be caught by the library and require
@@ -35,8 +38,8 @@ To understand the protocol in more depth, consult the
 
 ## Using this Library
 
-Start with the [build instructions](BUILD.md), then read guide for including
-this library as a [dependency to your project](DEPENDENCY.md).
+Start with the [build instructions](BUILD.md), then read the guide for including
+this library as a [dependency from your project](DEPENDENCY.md).
 
 Once you've all set up, you can take a look at our examples:
 
@@ -77,7 +80,7 @@ See a [complete example in C++](examples/line_sender_cpp_example.cpp).
 The API is sequentially coupled, meaning that methods need to be called in a
 specific order.
 
-For each row you need to specify a table name and at least one symbol or
+For each row, you need to specify a table name and at least one symbol or
 column. Symbols must be specified before columns.
 Once you're done with a row you must add a timestamp calling `at` or `at_now`.
 
@@ -101,7 +104,7 @@ a failure.
 You may then call `line_sender_error_msg(err)` and
 `line_sender_error_get_code(err)` to extract error details.
 
-Once handled, the error object *must* be disposed by calling
+Once handled, the error object *must* be disposed of by calling
 `line_sender_error_free(err)`.
 
 On error you must also call `line_sender_close(sender)`.