DA Lead Data Engineer
Core
Designing and implementing highly performant data ingestion pipelines, scalable frameworks, and end-to-end data pipelines to ensure data quality and consistency for healthcare diagnostics and drug development.
Role type
Lead Data Engineer
Builds
Data ingestion pipelines, scalable frameworks, and end-to-end data pipelines
Domain
Healthcare diagnostics and drug development
Deliverable
production ML models | infrastructure
Required skills
PySpark, Python, Scala, Snowflake, Databricks, ETL, Big Data modeling, CI/CD (Jenkins, Terraform, Docker, AWS, Kubernetes), Airflow, Hadoop, Kafka, PL/SQL, Data Warehousing concepts
Preferred skills
Pharma industry experience, Business intelligence, Tableau, Power BI, AWS EC2/S3/Lambda/SQS/SNS
Technologies
PySpark, Python, Scala, Snowflake, Databricks, Jenkins, Terraform, Docker, AWS, Kubernetes, Airflow, Hadoop, Kafka, Tableau, Power BI, PL/SQL
Responsibilities
Designing and implementing highly performant data ingestion pipelines from multiple sources; Developing scalable and re-usable frameworks for ingesting data sets; Integrating end-to-end data pipelines to ensure data quality and consistency; Working with event-based/streaming technologies to ingest and process data; Evaluating the performance and applicability of multiple tools against customer requirements
Seniority
Lead, hands-on IC with mentorship
