Senior Big Data Engineer
Core
Design, develop, and maintain sophisticated data pipelines, Ontology Objects, and Foundry Functions within Palantir Foundry to ensure end-to-end data integrity and support data-driven decision-making.
Role type
Senior IC data engineer (cloud & big data)
Builds
Scalable data pipelines, ETL/ELT processes, and data models for business applications
Domain
Life sciences / Pharmaceutical industry / Cloud data engineering
Deliverable
production ML models | product features
Required skills
Python, PySpark, SQL, TypeScript, Apache Spark, Apache Kafka, AWS Glue, Azure Data Factory, Google Cloud Dataflow, Docker, Kubernetes, data modeling, ETL/ELT concepts, database systems (PostgreSQL, MySQL, NoSQL)
Preferred skills
Apache Lucene, REST API, Veeva CRM, Reltio, SAP, Palantir Foundry, pharmaceutical industry regulations
Technologies
Palantir Foundry, Python, PySpark, SQL, TypeScript, Apache Hadoop, Apache Spark, Apache Kafka, BigQuery, AWS Glue, Azure Data Factory, Google Cloud Dataflow, PostgreSQL, MySQL, NoSQL, Docker, Kubernetes, Apache Lucene
Responsibilities
Design and implement scalable data pipelines in Palantir Foundry; Create data models, schemas, and flow diagrams; Develop and optimize ETL/ELT processes for data collection and integration; Monitor pipeline performance and implement improvements; Troubleshoot data pipeline issues; Stay current with emerging technologies and document technical solutions
Seniority
Senior, hands-on IC
