Senior Big Data Engineer
Core
Design, develop, and maintain scalable data processing solutions and optimize large-scale big data environments for Epsilon's TotalSource Plus demographic and lifestyle products.
Role type
Senior Big Data Engineer
Builds
Scalable, fault-tolerant data pipelines and optimized Spark jobs on Databricks
Domain
Marketing data / Big Data Engineering
Deliverable
production ML models
Required skills
Python, Apache Spark, Databricks, AWS, SQL, Distributed Systems, Data Modeling, Git, Unit Testing
Preferred skills
Scala, Java, Kubernetes, Docker, Airflow, MWAA, Machine Learning, Delta Lake, Unity Catalog
Technologies
Databricks, Apache Spark, AWS, Python, SQL, Kubernetes, Docker, Airflow, MWAA
Responsibilities
Design and implement scalable data pipelines; Optimize Spark jobs for performance; Develop and optimize SQL queries; Collaborate with cross-functional teams to translate requirements; Mentor junior engineers and conduct code reviews.
Seniority
Senior, hands-on IC