Technical Context & Problem Space
Build low-overhead Rust proxy dataplanes, WebSocket/gRPC bi-directional streaming routers, and cold-start weight streamers for multi-cloud GPU clusters.
Our Rust dataplane handles millions of concurrent inference streams, dynamically packing requests into GPU batches while streaming multi-GB model checkpoints from R2/S3 at 80+ Gbps per node. Every microsecond saved on the hot path translates directly into lower hardware footprint.
Key Engineering Deliverables
- Write custom Hyper/Tower load-shedding middleware, zero-copy tensor deserializers, and cold-start container warmup coordinators.
- Harden distributed state machines and I/O pipelines written in Tokio, Tonic, Hyper, Axum against network partitions and disk faults.
- Build deterministic benchmarks and chaos tests that catch regressions before production rollout.
Candidate Profile & Compensation
- 3+ years writing performance-critical systems software in Rust (with production exposure to Tokio and Tonic).
- Solid grasp of virtual memory, page caches, thread-per-core scheduling, or modern hardware interconnects.
- Annual base band: $185k - $250k USD (US Remote), paired with equity and async-first remote workflows.
