Technical Context & Problem Space
Lead open-source development on Text Generation Inference (TGI), Candle, Safetensors, and SIMD Rust tokenizers used across the AI ecosystem.
You will maintain zero-copy memory-mapped weight loading (Safetensors), multi-GPU tensor parallelism in Rust, and hardware-accelerated kernels across CUDA, Apple Metal, and wgpu. Every microsecond saved on the hot path translates directly into lower hardware footprint.
Key Engineering Deliverables
- Ship quantized GGUF/AWQ matrix multiplication kernels, maintain PyO3 wheels downloaded 50M+ times per month, and profile BPE tokenization throughput.
- Harden distributed state machines and I/O pipelines written in CUDA, PyO3, wgpu, Serde against network partitions and disk faults.
- Build deterministic benchmarks and chaos tests that catch regressions before production rollout.
Candidate Profile & Compensation
- 3+ years writing performance-critical systems software in Rust (with production exposure to CUDA and PyO3).
- Solid grasp of virtual memory, page caches, thread-per-core scheduling, or modern hardware interconnects.
- Annual base band: $185k - $240k USD (Worldwide Remote), paired with equity and async-first remote workflows.
