Data Engineer
Core
Design, build, maintain, and optimize large-scale data pipelines and ETL/ELT processes to generate actionable insights and support business decisions.
Role type
Senior Data Engineer
Builds
Scalable data pipelines, data models, and integration solutions for structured and unstructured data
Domain
Big Data, Cloud Data Platforms, ETL/ELT
Deliverable
production ML models
Required skills
PySpark, SQL, Databricks, Apache Spark, ETL/ELT, Data Modeling, Cloud Platforms (AWS), Workflow Orchestration, Data Governance, Performance Tuning
Preferred skills
Python, R, SageMaker, Apache Airflow, Data Visualization, GDPR/CCPA compliance, Machine Learning model development
Technologies
Databricks, PySpark, SQL, AWS, Apache Spark, Apache Airflow, SageMaker
Responsibilities
Design and develop data pipelines leveraging Databricks, PySpark, and SQL; Engineer solutions for both structured and unstructured data; Implement automated workflows for data ingestion, transformation, and deployment; Apply performance optimization techniques including Spark job tuning; Build integrations with multiple data sources; Collaborate with cross-functional teams to understand data requirements; Develop and maintain data models and documentation; Implement data security and privacy measures.
Seniority
Mid-Senior, hands-on IC