AI Research Engineer - Datadog AI Research (DAIR)
Core
Build data pipelines, training infrastructure, and simulation environments to turn research ideas into working systems for AI-powered observability and security.
Role type
Senior IC research engineer (distributed systems & RL infra)
Builds
Multimodal foundation models, trained autonomous agents, and evaluation infrastructure for cloud observability
Domain
Cloud observability, security, and distributed systems
Deliverable
production ML models
Required skills
distributed computing, RL infrastructure, ML systems for training/inference, Python, systems programming (Rust/C++/Go), large-scale model training/fine-tuning, GPU programming
Preferred skills
observability/SRE/security domain knowledge, bridging research to product, CUDA optimization, production data pipelines
Technologies
Ray, Slurm, PyTorch, JAX, Megatron-LM, DeepSpeed, SkyRL, VeRL, TorchTitan, CUDA
Responsibilities
Build and operate multimodal data pipelines and training/evaluation infrastructure; Implement models and run experiments at scale; Build simulation environments and replay infrastructure for agent training; Orchestrate distributed training and RL with Ray; Establish automated benchmarks and regression tests; Collaborate to integrate capabilities into products and harden prototypes
Seniority
Senior, hands-on IC
