Systems Architecture at Anthropic
Our inference runtime disaggregates prefill and decode clusters across thousands of accelerators connected via RoCEv2 and NVLink. The Rust control plane manages microsecond-level request routing, prefix-tree KV-cache eviction, continuous batching, and zero-copy tensor transfer over GPUDirect RDMA. Why Rust? Python GIL contention stalls token scheduling when batch step times drop below 8ms.
Architect low-latency distributed token generation pipelines, disaggregated KV-cache engines, and high-throughput Rust control planes for Claude frontier models. Base pay for this remote role sits at $240,000 to $320,000 USD plus equity.
What You Will Own in Production
- You will debug multi-node NCCL/RDMA collective stalls, write lock-free RadixAttention prefix routers in Tokio, and interface directly with custom FP8 CUDA kernels via C FFI.
- Profile CPU cache misses, syscall bottlenecks, and tail latency spikes across Tokio, CUDA, Tonic services using hardware perf counters and flamegraphs.
- Review architectural RFCs and maintain strict memory-safety invariants in async and unsafe Rust blocks.
Background We Look For
- Production track record shipping low-level systems in Rust, C, or C++ with deep fluency in Tokio, CUDA, Tonic, PyO3, io_uring.
- Hands-on experience debugging Linux kernel networking, NVMe storage, or GPU compute pipelines under heavy load.
- Compensation: $240,000 to $320,000 USD per year, equity grant, and full remote workstation budget (San Francisco / US Remote).
