Senior Data Engineer
Core
Design, develop, and optimize complex data pipelines, integration frameworks, and metadata-driven architectures to enable seamless data access and analytics for enterprise-wide data management.
Role type
Senior IC data engineer (big data & distributed computing)
Builds
Scalable ETL/ELT pipelines, real-time and batch data processing solutions, CI/CD pipelines for data deployments, and data virtualization layers.
Domain
Biotechnology / Pharma / Enterprise Data Fabric
Required skills
Databricks, PySpark, SparkSQL, AWS services, Python, SQL, workflow orchestration, performance tuning on big data, data modeling, data governance, CI/CD practices (via careerplan.io/jobs/R-213363-senior-data-engineer-at-amgen)
Preferred skills
Data Mesh architecture, API development, vector databases for LLMs, OLAP/OLTP database tuning, Git version control, automated unit testing
Technologies
Apache Spark, Hadoop, Databricks, AWS, Jenkins, Maven, SQL, NOSQL, Git
Responsibilities
Design and maintain scalable ETL/ELT pipelines for structured, semi-structured, and unstructured data; Implement real-time and batch data processing solutions integrating multiple sources; Optimize big data processing frameworks for high availability and cost efficiency; Work with metadata management and data lineage tracking tools; Ensure data security, compliance, and role-based access control; Optimize query performance, indexing, partitioning, and caching for large-scale datasets; Develop CI/CD pipelines for automated data pipeline deployments; Implement data virtualization techniques; Collaborate with data architects, business analysts, and DevOps teams.
Seniority
Senior, hands-on IC
