Neon (Databricks) • 📍 Worldwide Remote •
$190k - $265k USD
Improve L0-to-L1 compaction throughput over `tokio-epoll-uring`, lower tail latency on cold page reconstructions, and harden Paxos WAL quorum recovery.
#lsm-tree • $180k-$243k USD Base
LSM-Tree in Production Rust: Architecture & Open Roles
Log-Structured Merge-tree storage engines optimized for write throughput & S3
1. Why Teams Deploy LSM-Tree
Log-Structured Merge-trees (LSM-Trees) convert random disk writes into sequential append-only operations, making them the architecture of choice for write-heavy databases, stream processors, and cloud-native storage layers. In the Rust ecosystem, teams build disaggregated LSM engines over S3, such as Neon's Pageserver, RisingWave's Hummock, and SlateDB, or use pure-Rust embedded LSMs like `fjall`.
2. LSM-Tree Kernel & Memory Internals
Incoming writes append to a Write-Ahead Log (WAL) and insert into an in-memory `MemTable` (typically a lock-free concurrent skip list). Once full, the MemTable flushes to immutable Level-0 SSTables (Sorted String Tables) on NVMe or S3. Background compaction threads merge overlapping SSTables across tiers, using Bloom filters and block caches (`foyer`) to bound read amplification.
3. LSM-Tree Interview & Hiring Bar
Storage engine interviews focus heavily on write/read/space amplification trade-offs, Leveled vs. Universal compaction strategies, and tuning compaction I/O throttling to prevent p99 latency spikes.
Open LSM-Tree Engineering Positions (4)
Turso • 📍 Worldwide Remote •
$175k - $235k USD
Implement SQLite bytecode VDBE instructions in safe Rust, expand deterministic simulation fuzzing seeds, and reduce page-cache lock contention.
Materialize • 📍 US / Canada Remote •
$190k - $255k USD
Reduce arrangement memory footprints, optimize multi-way stream join planning, and tune S3 Parquet compaction inside the Persist storage engine.
RisingWave Labs • 📍 Worldwide Remote •
$165k - $215k USD
Tune hybrid memory+NVMe cache eviction in `foyer`, reduce S3 API request costs during streaming barrier checkpoints, and speed up state recovery.