Security Infrastructure and Platform Isolation at Databricks
Databricks coordinates compute clusters running millions of concurrent analytics and machine learning jobs. Every compute node must authenticate against the control plane without introducing millisecond-level latency to distributed query execution. Scale is immense. We build decentralized security sidecars in Rust that handle SPIFFE and SPIRE workload attestation and cryptographic token verification at line speed. Rather than relaying tokens across user space network stacks, our agents hook socket operations via eBPF sockops programs. Latency drops to sub-microsecond levels. Precision matters. You will design distributed key rotation protocols and high-throughput policy evaluation engines that secure thousands of tenant clusters while running under stringent CPU and memory caps.
Disclosed base compensation for this role ranges from $228,000 to $303,000 USD annually, accompanied by equity and bonuses. Total rewards competitive.
Core Engineering Responsibilities
- Develop lightweight zero-trust authentication proxies in Rust deployed alongside distributed query engines.
- Implement kernel-level packet inspection and TLS handoff routines using eBPF and XDP drivers. Linux networking depth required.
- Manage cryptographically signed workload identities using SPIFFE across multi-cloud infrastructure.
- Collaborate with database kernel teams to ensure security checks never bottleneck data lakehouse throughput.
Technical Requirements
- Extensive experience with systems programming in Rust or C++ in distributed environments.
- Deep knowledge of transport layer security (TLS 1.3), public key cryptography, and identity federation.
- Practical familiarity with Linux networking internals, eBPF, and kernel socket architectures.
- Disclosed compensation: $228,000 to $303,000 USD base salary plus equity.
