Senior Data Engineer
Core
Build and maintain ML/AI data infrastructure powering an email security platform to enable threat detection and investigation.
Role type
Senior IC data engineer (ML infrastructure)
Builds
Scalable data pipelines, feature engineering frameworks, and offline/online feature stores for ML models and LLM-powered agents.
Domain
Cybersecurity / Email security / Machine Learning infrastructure
Deliverable
production ML models
Required skills
Distributed data systems, high-scale data pipelines, Python, Infrastructure-as-Code (Terraform), open table formats (Apache Iceberg, Hudi, DeltaLake), data modeling, ETL/ELT pipelines, workflow orchestration (Apache Airflow), observability
Preferred skills
Feature engineering frameworks and feature stores (Feast, Tecton), Kubernetes, data quality frameworks
Technologies
AWS, Azure, GCP, Spark, Flink, Dask, Ray, Beam, DataBricks, Athena, Glue, Kubernetes, Iceberg, Hudi, DeltaLake, Airflow
Responsibilities
Develop and maintain scalable data pipelines on AWS/Azure using Spark, Airflow, Athena, Kubernetes; Design and optimize Iceberg-based data lake tables; Build and manage feature engineering frameworks for batch and real-time feature serving; Develop and maintain training data pipelines optimized for distributed ML model training; Collaborate with data scientists and security researchers to translate data requirements into production-grade solutions; Monitor and optimize data pipeline performance; Mentor junior engineers
Seniority
Senior, hands-on IC with mentorship
