Data Engineer
Core
Design and implement big data solutions on a scalable cloud platform, processing batch and streaming data to derive advanced insights for end users.
Role type
Data Engineer
Builds
Big data solutions, batch processing pipelines, streaming ETL/ELT processes, APIs, and monitoring dashboards
Domain
Cloud data engineering, big data analytics
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Apache Spark, Python, SQL, Databases, Azure Data Lake Storage, Delta Live Tables
Preferred skills
Databricks, FastAPI, Docker, AWS, GCP
Technologies
Apache Spark, Python, Scala, SQL, Azure, Delta Live Tables, Databricks, FastAPI, Docker, AWS, GCP
Responsibilities
Design and implement batch processing pipelines; Design and implement streaming ETL/ELT processes; Write code for complete data integration or analytic use cases; Design and implement APIs; Write end-to-end and functional tests; Implement monitoring solutions with alerts and dashboards
Seniority
Mid-level, hands-on IC
