Staff ML Engineer, Perception Research
Core
Develop and deploy state-of-the-art Multimodal LLMs and World models for 3D Perception using Camera, LiDAR, and Radar sensor data to advance autonomous driving capabilities.
Role type
Staff ML Engineer (Perception Research)
Builds
Multimodal LLMs, World models, and scalable inference pipelines for Waymo's autonomous driving system.
Domain
Autonomous driving, Computer Vision, Reinforcement Learning, Foundation Models
Deliverable
production ML models
Required skills
Multimodal LLM development, World models, 3D Perception, Reinforcement Learning, Foundation Models, Model optimization (quantization, pruning, distillation), Distributed training (Data parallel, FSDP), Transformer architecture optimization, Scalable data pipelines, Evaluation framework design, Performance profiling
Preferred skills
Generative models (diffusion, autoregressive), Framework contributions (JAX, Gemax, XManager), High-impact industry AI projects
Technologies
JAX, Flax, TensorFlow, PyTorch, xprof
Responsibilities
Conduct comprehensive experimentation to train and deploy Multimodal LLMs and World models for 3D Perception; Partner with engineering and research teams to deploy models and implement efficient workflows; Apply techniques like quantization, pruning, and knowledge distillation; Develop scalable data pipelines for training and evaluation; Design evaluation frameworks for perception models; Build infrastructure for large-scale model distillation and bulk-inference; Experiment with model partitioning and sharding strategies; Build tools for performance analysis and debugging.
Seniority
Staff, hands-on IC with strategic impact