Data Engineering Professional II
Core
Design, develop, and maintain scalable data pipelines and ETL/ELT processes for structured and unstructured data within a regulated pharma environment.
Role type
Senior Data Engineer (MDM & Cloud)
Builds
Scalable data pipelines, optimized ETL/ELT processes, and lakehouse architectures on Azure/Databricks.
Domain
Pharmaceutical / Healthcare / Data Engineering (via careerplan.io/jobs/R0183785-data-engineering-professional-ii-at-takeda)
Required skills
Databricks (Spark, PySpark, Delta Lake), Python, SQL, Azure, ETL/ELT design, CI/CD, data governance, GxP compliance
Preferred skills
AWS, Airflow, distributed data processing, mentoring, regulated environment experience
Technologies
Databricks, Spark, PySpark, Delta Lake, Azure, AWS, Airflow, S3, Lambda
Responsibilities
Design and maintain scalable data pipelines; Build and optimize ETL/ELT processes; Work with Delta Lake and lakehouse architecture; Ensure data quality, integrity, and compliance with governance and GxP standards; Collaborate with data scientists and business stakeholders; Implement CI/CD, version control, and automation best practices; Monitor, troubleshoot, and optimize data pipelines.
Seniority
Senior, hands-on IC