Skip to content

Latest commit

 

History

History
97 lines (69 loc) · 3.75 KB

CONTRIBUTING.md

File metadata and controls

97 lines (69 loc) · 3.75 KB

Using the dev container

The devcontainer comes preloaded with all of the requisite dependencies. It includes a Makefile to generate bindings.
Use make generate-bindings to generate everything. Use make generate-bridge-bindings for the bridge code. Use make generate-kernel-bindings for the kernel code

Rebuilding Rust extension and interop layer

To regen core interop from header, install ClangSharpPInvokeGenerator like:

dotnet tool install --global ClangSharpPInvokeGenerator

Then, run:

ClangSharpPInvokeGenerator @src/DeltaLake/Bridge/GenerateInterop.rsp

The Rust DLL is built automatically when the project is built.

dotnet tool install --global ClangSharpPInvokeGenerator

Rebuilding Kernel FFI and interop layer

delta-kernel-rs is linked as a git submodule in this repo.

We first build the Kernel DLL and FFI header, and use the header to generate the C# Interop classes.

All 3 of these things are tightly coupled and must happen sequentially when you git clone this repo:

  1. The git submodule must be initiated
  2. The dll and .h must be built
  3. The .h header is used to generate the interop .cs classes

Note that the .dll/.so/... files are continuously generated via the csproj - but the .h and the .cs files are git committed, and must be generated by a maintainer using the steps below, whenever we bump the FFI version in delta-kernel-rs.version.txt.

Build Kernel DLL

$GIT_ROOT = git rev-parse --show-toplevel
cd "$GIT_ROOT"

# Initiate submodule at release tag
#
git submodule update --init
$DELTA_KERNEL_RS_TAG = Get-Content $GIT_ROOT\src\DeltaLake\Kernel\delta-kernel-rs.version.txt
git -C $GIT_ROOT\src\DeltaLake\Kernel\delta-kernel-rs checkout $DELTA_KERNEL_RS_TAG

# Build the FFI with cloud storage features
#
$FFI_PROJ_PATH = "$GIT_ROOT/src/DeltaLake/Kernel/delta-kernel-rs/ffi"
cargo build --manifest-path $FFI_PROJ_PATH/Cargo.toml --no-default-features --features "default-engine-rustls"

Build C# Interop

# Copy the built header from cargo into C# Interop project
#
$HEADER_FILE = "delta_kernel_ffi.h"
$GENERATED_FFI_HEADER = "${GIT_ROOT}/src/DeltaLake/Kernel/delta-kernel-rs/target/ffi-headers/${HEADER_FILE}"
$CSHARP_FRIENDLY_FFI_HEADER = "$GIT_ROOT/src/DeltaLake/Kernel/include/${HEADER_FILE}"

Copy-Item -Path $GENERATED_FFI_HEADER -Destination $CSHARP_FRIENDLY_FFI_HEADER -Force

ClangSharpPInvokeGenerator -D DEFINE_DEFAULT_ENGINE_BASE=1 @src/DeltaLake/Kernel/GenerateInterop.rsp
Alternatively, if you have an environment that is not particularly cooperative, you can use the included
docker container and run 
```bash
docker run --mount type=bind,src=.,dst=/app clangsharp -D DEFINE_DEFAULT_ENGINE_RUSTLS=1 @src/DeltaLake/Kernel/GenerateInterop.rsp

Post-processing script to remove Mangled entrpoints - ClangSharpPInvokeGenerator

does not seem to have a built-in arg that does this.

Get-ChildItem "$GIT_ROOT/src/DeltaLake/Kernel/Interop" -Filter Interop.cs -Recurse | ForEach-Object { $content = Get-Content $.FullName -Raw $updatedContent = $content -replace ', EntryPoint = "[^"]+"', '' Set-Content -Path $.FullName -Value $updatedContent }


### Regenerating API docs

Install [docfx](https://dotnet.github.io/docfx/), then run:

    docfx src/DeltaLake.ApiDoc/docfx.json