Serverless ML Infrastructure at Modal
Agent execution environments demand sub-second container instantiation across GPU nodes. Modal eliminates virtualization latency by scheduling user containers directly over custom Linux kernel namespaces and lightweight isolation monitors. Container cold starts kill interactivity. Our execution plane pre-warms PyTorch runtime environments in host memory, routing incoming model serving payloads through zero-copy ring buffers. GPU device memory is shared dynamically across tenant workloads without driver resets. This keeps agent tool calls responsive during bursty production traffic.
As a Forward Deployed ML Systems Engineer, you collaborate directly with customers scaling frontier coding agents and real-time generation models. Base compensation ranges from $180,000 to $250,000 USD plus equity.
What You Will Work On
- Optimize distributed model weight streaming to initialize large language models in tens of milliseconds.
- Troubleshoot CUDA driver context initialization, pinned memory pools, and PCIe bus contention during concurrent inference.
- Profile end-to-end latency for autonomous coding agent sandboxes requiring rapid container creation and teardown.
- Harden container boundary isolation and network egress policies using custom Rust supervisors and Linux namespaces.
Background We Seek
- Strong software engineering background in Rust or Python, with direct systems-level debugging experience on Linux.
- Hands-on familiarity with deep learning serving stacks (vLLM, TensorRT-LLM, Hugging Face) and GPU memory management.
- Comfortable diagnosing kernel traces, memory fragmentation, and distributed network latency under real production workloads.
- Salary: $180,000 to $250,000 USD base compensation with equity ownership and comprehensive health coverage.
