Data Engineer, Amazon Traffic Engineering
Core
Build and operate production-grade data pipelines and feature infrastructure that ingest billions of events to support ML and Science initiatives for Bot Management.
Role type
Senior IC data engineer (ML infrastructure)
Builds
Production data pipelines, versioned feature groups, and a Feature Store for bot detection models
Domain
E-commerce security / Bot Management / Machine Learning
Deliverable
production ML models
Required skills
Data modeling, ETL pipeline construction, large-scale high-throughput system design, Python/Java/Scala/NodeJS, batch and streaming architectures (Kafka, Kinesis, Flink), software engineering best practices
Preferred skills
AWS technologies (Redshift, S3, Glue, EMR, Lambda), non-relational databases (object, document, graph, column-family)
Technologies
AWS Glue, S3, OpenSearch, Apache Flink, Kinesis, Kafka
Responsibilities
Build and own batch and near real-time pipelines for Trails and Non-Trails sources; Develop versioned feature groups across multiple storage backends; Operate streaming systems for low-latency feature computation; Partner with Applied Scientists to define data contracts and SLAs; Own reliability, monitoring, and cost efficiency of pipelines
Seniority
Senior, hands-on IC
