Senior Data Engineer
Core
Design, build, and operate large-scale batch and real-time data pipelines powering enterprise data and analytics platforms.
Role type
Senior IC data engineer
Builds
Enterprise data and analytics platforms
Domain
Enterprise data engineering and cloud-native data platforms
Deliverable
production ML models | product features | infrastructure
Required skills
distributed data processing, event streaming, workflow orchestration, cloud data warehousing, secrets management, distributed systems architecture
Preferred skills
data observability, pipeline health monitoring, SLA management
Technologies
Kafka, Spark, EMR, Hadoop, Java, Python, SQL, Airflow, Control-M, Autosys, Step Functions, AWS, Azure, GCP
Responsibilities
Design and develop high-volume batch and real-time data pipelines; Build end-to-end data solutions covering ingestion, transformation, orchestration, streaming, and downstream delivery; Develop event-driven and real-time data processing solutions; Build scalable data processing solutions using distributed computing frameworks; Implement and manage workflow orchestration and scheduling for complex data pipelines; Work with cloud data warehouse and cloud-native data platforms to support enterprise-scale workloads; Design solutions with a strong focus on reliability, scalability, performance, security, and low latency; Implement secure approaches for secrets management, credentials, and service-to-service authentication in production environments.
