Systems Architecture at Together AI
Together AI runs tens of thousands of GPUs across diverse cluster fabrics. Milliseconds matter here. The compute control plane uses declarative state reconciliation loops to orchestrate cold-starts, pre-warmed model weights, dynamic LoRA adapters, and multi-tenant isolation. Fast convergence is mandatory. Writing controller logic with Tokio and Tonic guarantees zero runtime garbage collection pauses and predictable RPC latency under burst traffic.
Design the declarative Kubernetes-native control plane orchestrating Together AI's global GPU inference fleet and multi-accelerator serving runtime. Base pay for this remote role sits at $200,000 to $280,000 USD plus equity.
What You Will Own in Production
- Reconcile thousands of GPU worker nodes, write high-concurrency custom controllers in Tokio, and enforce sub-100ms cluster state reconciliation.
- Profile CPU cache misses, syscall bottlenecks, and tail latency spikes across Tokio, Kubernetes, CUDA services using hardware perf counters and flamegraphs.
- Review architectural RFCs and maintain strict memory-safety invariants in async and unsafe Rust blocks.
Background We Look For
- Production track record shipping low-level systems in Rust, C, or C++ with deep fluency in Tokio, Kubernetes, CUDA, vLLM, Tonic.
- Hands-on experience debugging Linux kernel networking, NVMe storage, or GPU compute pipelines under heavy load.
- Compensation: $200,000 to $280,000 USD per year, equity grant, and full remote workstation budget (San Francisco / US Remote).
