Add Julia native arrow flight support

[GTrunSec]

This PR introduces experimental Arrow Flight support to Arrow.jl and follows up with a production-focused pass on Arrow write-path performance, interop coverage, and package modularization.

The Flight work adds an in-tree client implementation, IPC conversion helpers, interoperability coverage against pyarrow.flight, and an optional native Julia server bridge. The performance work focuses on reducing conversion and write overhead for common high-throughput serialization shapes, especially offset-backed inputs, nested lists, maps, strings, binaries, and Any-typed wrappers.

What Changed

Arrow Flight

• Added experimental Arrow.Flight support in-tree
• Vendored and generated Arrow Flight protocol bindings from Flight.proto
• Added client constructors for the full FlightService RPC surface:
  • Handshake
  • ListFlights
  • GetFlightInfo
  • PollFlightInfo
  • GetSchema
  • DoGet
  • DoPut
  • DoExchange
  • DoAction
  • ListActions
• Added high-level FlightData <-> Arrow IPC conversion helpers
• Added request-header, binary-metadata, token-reuse, and TLS helpers
• Added a transport-agnostic Flight server core for local dispatch and testing
• Added an optional gRPCServer.jl extension for native Julia server integration
• Added targeted CI coverage for Flight interoperability

Interoperability and Test Coverage

• Added live interoperability coverage against pyarrow.flight for:
  • discovery APIs
  • DoGet
  • DoPut
  • DoExchange
  • DoAction
  • ListActions
  • Handshake
  • authenticated token propagation
  • TLS
  • PollFlightInfo
• Kept the Flight test surface modular under test/flight/
• Added a temporary-environment runner for optional gRPCServer.jl coverage without mutating the shared test environment

ArrowTypes and Serialization Behavior

• Added built-in Complex roundtrip support through ArrowTypes extension metadata
• Raised ArrowTypes compat to Julia 1.12
• Unified offset-axis handling around Julia 1.12 APIs
• Refined ToArrow source access, conversion, iteration, and materialization behavior

Write-Path Performance

• Added fast paths for:
  • offset-backed primitive wrappers
  • nullable primitive validity scanning
  • bool bit packing
  • contiguous primitive ToArrow writes
  • ToList{UInt8} string/binary chunk writes
  • fixed-size tuple-backed writes
  • map flatten/build paths
• Added targeted direct Arrow.tobuffer fast paths for:
  • eligible single-partition inputs
  • targeted multi-partition single-column string streams
  • targeted multi-partition single-column non-missing binary streams
• Intentionally kept broader cases on the legacy writer path unless the benchmark signal was stable

Tooling and Hygiene

• Restored RAT/license cleanliness for newly added files
• Added and used benchmark harnesses to gate performance-sensitive changes
• Kept the Flight implementation and tests modular instead of expanding monolithic files

Performance Notes

Local warm write-path benchmarks show substantial improvements on several production-relevant shapes. Representative results from the current branch include:

• offset_map: 11.238 ms -> 2.312 ms
• offset_nested_list: 2.238 ms -> 0.797 ms
• offset_binary: 2.228 ms -> 1.818 ms
• offset_string: 1.829 ms -> 1.738 ms

In addition, several previously problematic Any-typed offset-backed cases are now stable and writable.

These numbers are local throughput indicators, not a formal cross-machine performance contract.

Validation

• Pkg.test("Arrow")
• Pkg.test() in src/ArrowTypes
• targeted Flight suite
• live pyarrow.flight interoperability coverage
• run_rat.sh .
• git diff --check

Notes

• Flight support is intentionally marked experimental
• Native Julia Flight server transport remains optional/experimental
• Direct multi-partition stream optimizations were kept narrow on purpose; broader widenings were only retained when benchmark and package-test results stayed positive