LanceDB • 📍 US / Canada Remote •
$180k - $245k USD
Implement custom DataFusion physical execution nodes, disk-based IVF-PQ vector indices, and high-concurrency S3 byte-range schedulers.
#apache-arrow • $178k-$237k USD Base
Apache Arrow in Production Rust: Architecture & Open Roles
Zero-copy in-memory columnar format & SIMD compute kernels
1. Why Teams Deploy Apache Arrow
Apache Arrow (`arrow-rs`) has triggered a complete rewrite of the modern analytical database stack in Rust. Rather than serializing rows into protobufs or JSON, Arrow defines a standardized, cache-aligned columnar memory layout. Engines like InfluxDB 3.0, Polars, LanceDB, and Databend exchange gigabytes of tabular and vector data across process boundaries with zero deserialization CPU cost.
2. Apache Arrow Kernel & Memory Internals
The `arrow-array` and `arrow-buffer` crates represent nullable typed columns using separate validity bitmaps, offset buffers, and contiguous value slices aligned to 64-byte boundaries. Because values sit contiguously in RAM, the compiler auto-vectorizes filter, sum, and comparison kernels into AVX-512 or ARM NEON instructions. Arrow Flight (`arrow-flight`) extends this zero-copy principle over the network via gRPC streams.
3. Apache Arrow Interview & Hiring Bar
Data infrastructure and lakehouse vendors pay $185k to $260k+ for engineers fluent in `arrow-rs`. Interviews focus on `RecordBatch` memory lifecycles, dictionary encoding, and writing vectorized compute kernels without bounds-check penalties.
Open Apache Arrow Engineering Positions (5)
Polars • 📍 Worldwide Remote •
$170k - $225k USD
Optimize out-of-core streaming hash joins, cardinality estimation in the lazy query planner, and Parquet predicate pushdown.
Materialize • 📍 US / Canada Remote •
$190k - $255k USD
Reduce arrangement memory footprints, optimize multi-way stream join planning, and tune S3 Parquet compaction inside the Persist storage engine.
InfluxData • 📍 US / Europe Remote •
$185k - $245k USD
Contribute upstream to Apache DataFusion, reduce memory pressure during wide time-series compactions, and optimize Arrow Flight SQL throughput.
RisingWave Labs • 📍 Worldwide Remote •
$165k - $215k USD
Tune hybrid memory+NVMe cache eviction in `foyer`, reduce S3 API request costs during streaming barrier checkpoints, and speed up state recovery.