Data Platform Engineer
Core
Design and build scalable data pipelines, ETL/ELT processes, and data lakes on major cloud platforms.
Role type
Senior Data Platform Engineer
Builds
Scalable data pipelines, ETL/ELT processes, data lakes, and real-time streaming solutions
Domain
Cloud Data Engineering
Required skills
PySpark, Spark, Python, SQL, AWS/Azure/GCP, Apache Airflow, Kafka, dbt, CI/CD, DevOps
Preferred skills
Informatica, Talend, Matillion, Parquet, Avro, JSON, Big Data architectures
Technologies
Databricks, PySpark, Spark, AWS, Azure, GCP, S3, Blob Storage, BigQuery, Redshift, Apache Airflow, Kafka, dbt
Responsibilities
Build and maintain scalable data pipelines for large-scale data processing; Develop and implement ETL/ELT processes using PySpark and Spark; Design and deploy solutions in major cloud platforms; Support development of Big Data processing frameworks and data lakes; Implement data ingestion strategies for secure and efficient data movement; Work on real-time streaming and batch data processing; Contribute to performance tuning of Spark jobs; Maintain governance and data security practices.
Seniority
Senior, hands-on IC