Anthropic • 📍 San Francisco / US Remote •
$240k - $320k USD
You will debug multi-node NCCL/RDMA collective stalls, write lock-free RadixAttention prefix routers in Tokio, and interface directly with custom FP8 CUDA kernels via C FFI.
#cuda • $204k-$274k USD Base
CUDA in Production Rust: Architecture & Open Roles
GPU tensor kernels, PagedAttention & RDMA orchestration from Rust
1. Why Teams Deploy CUDA
Modern LLM inference and zero-knowledge proof generation are bound by HBM memory bandwidth and multi-GPU interconnect latency. While low-level matrix multiplication kernels (`cutlass`, FlashAttention-3) are written in CUDA C++ or Triton, AI infrastructure companies like Anthropic, Together AI, Fireworks AI, and Hugging Face use Rust (`cudarc`, Candle) to orchestrate GPU streams, KV-cache block tables, and RDMA tensor transfers.
2. CUDA Kernel & Memory Internals
A Python control loop struggles to feed 8x H100 or B200 GPUs when token generation steps take less than 10 milliseconds, because Python GIL contention leaves GPU streaming multiprocessors idle between kernel launches. A Rust inference scheduler manages CUDA stream events via FFI (`cudarc`), executes prefix-aware continuous batching, and orchestrates zero-copy NCCL / GPUDirect RDMA transfers across InfiniBand.
3. CUDA Interview & Hiring Bar
AI infrastructure engineers combining Rust systems mastery with CUDA memory hierarchy knowledge (HBM vs. SRAM shared memory, warp scheduling, FP8 quantization) command the highest compensation in the market ($220k to $320k+).
Open CUDA Engineering Positions (5)
Together AI • 📍 US / EU Remote •
$195k - $265k USD
Work directly on speculative decoding tree verification, PyO3 bindings for custom CUDA graphs, and Tonic bi-directional gRPC streaming backpressure.
Hugging Face • 📍 Worldwide Remote •
$185k - $240k USD
Ship quantized GGUF/AWQ matrix multiplication kernels, maintain PyO3 wheels downloaded 50M+ times per month, and profile BPE tokenization throughput.
CoreWeave • 📍 US Remote / East Coast •
$215k - $295k USD
Develop low-overhead Rust telemetry agents, automated topology-aware slurm/Kubernetes job placement plugins, and RDMA congestion diagnostics.