Engineering Overview: Staff Rust Distributed Search & Log Engine Engineer (Quickwit / Tantivy)
Quickwit decouples stateless Rust searchers from S3 split files, utilizing byte-range coalescing and SIMD bitpacking in Tantivy to query 10+ TB of logs in under 800ms. Advance Tantivy columnar fastfields, optimize Chitchat gossip cluster membership, and scale zero-disk Kafka-to-S3 indexing pipelines.
Responsibilities
- Design and maintain high-throughput Rust subsystems centered on Tantivy, Parquet, Apache Arrow.
- Eliminate mutex contention and unnecessary serialization copies across internal RPC and storage boundaries.
- Collaborate asynchronously with senior systems architects across US / EU Remote.
Qualifications & Total Rewards
- Proven systems engineering background in Rust, including deep knowledge of Parquet, Apache Arrow, Tokio.
- Ability to reason from first principles about CPU cache lines, I/O bandwidth, and distributed consensus trade-offs.
- Transparent pay band: $200,000 to $275,000 USD base salary, equity ownership, and hardware stipend.
