Data Platform Engineer
Core
Design and build scalable data pipelines, ETL/ELT processes, and data lakes on major cloud platforms to support large-scale data processing and real-time streaming.
Role type
Senior Data Platform Engineer
Builds
Scalable data pipelines, data lakes, and cloud-based data processing frameworks
Domain
Cloud Data Engineering / Big Data
Deliverable
production ML models | product features | infrastructure
Required skills
PySpark, Spark, Python, SQL, Cloud platforms (AWS/Azure/GCP), Data orchestration (Airflow), Data formats (Parquet/Avro/JSON), Troubleshooting, Performance optimization
Preferred skills
Kafka, CI/CD, DevOps, dbt, Informatica/Talend/Matillion
Technologies
Databricks, PySpark, Spark, AWS, Azure, GCP, S3, Blob Storage, BigQuery, Redshift, Apache Airflow, Kafka, dbt
Responsibilities
Develop and implement ETL/ELT processes using PySpark and Spark; Design and deploy solutions in major cloud platforms; Support development of Big Data processing frameworks and data lakes; Implement data ingestion strategies for secure and efficient data movement; Work on real-time streaming and batch data processing; Contribute to performance tuning of Spark jobs; Maintain governance and data security practices
Seniority
Senior, hands-on IC