Senior Data Engineer
Core
Design, develop, and optimize complex data pipelines, integration frameworks, and metadata-driven architectures to enable seamless data access and analytics across the Enterprise Data Fabric.
Role type
Senior Data Engineer
Builds
Scalable ETL/ELT pipelines, real-time and batch data processing solutions, CI/CD pipelines, and data virtualization layers.
Domain
Biotechnology / Big Data Engineering
Deliverable
production ML models
Required skills
Databricks, PySpark, SparkSQL, AWS, Python, SQL, Workflow Orchestration, Data Fabric, Data Mesh, CI/CD, Git
Preferred skills
Biotech & Pharma domain knowledge, API development, Vector databases for LLMs, OLAP/OLTP data modeling, Jenkins, Maven, Automated unit testing
Technologies
Apache Spark, Hadoop, AWS, Databricks, Jenkins, Maven, Git, SQL, NOSQL, Vector databases
Responsibilities
Design and maintain scalable ETL/ELT pipelines; Implement real-time and batch data processing solutions; Optimize big data processing frameworks; Work with metadata management and data lineage tracking; Ensure data security and compliance; Optimize query performance and indexing; Develop CI/CD pipelines for automated deployments; Implement data virtualization techniques.
Seniority
Senior, hands-on IC