Inside the Stack (LSM-Tree / Parquet / Apache Arrow)
Materialize runs Rust where garbage pauses and unpredictable memory growth are unacceptable. Materialize compiles arbitrary SQL queries into Timely & Differential Dataflow data-parallel graphs backed by a cloud-native LSM storage layer (Persist) over S3.
Core Challenges on Your Plate
- Reduce arrangement memory footprints, optimize multi-way stream join planning, and tune S3 Parquet compaction inside the Persist storage engine.
- Scale internal components leveraging LSM-Tree, Parquet, Apache Arrow, Tokio, Prost while keeping binary compile times and memory RSS tight.
- Instrument p99.9 latency metrics and lead root-cause incident analyses across distributed clusters.
Requirements & Verified Salary Band
- Deep familiarity with idiomatic Rust, async execution models, and systems debugging tools (`gdb`, `perf`, `ebpf`).
- Prior ownership of production storage engines, compilers, network proxies, or cryptographic runtimes.
- Base salary: $190,000 to $255,000 USD + equity. Location: US / Canada Remote.
