Data Engineer
Core
Design, build, and maintain scalable data infrastructure and pipelines to ensure high data availability, quality, and reliability.
Role type
Data Engineer
Builds
Scalable data pipelines, ETL processes, and data solutions for data scientists and business stakeholders
Domain
Data Engineering / Data Warehousing
Deliverable
production ML models | infrastructure
Required skills
Python, SQL, data warehousing (Amazon Redshift), relational databases (PostgreSQL, MySQL, SQL Server), big data technologies (Hadoop, Spark, Kafka), data modeling, database design
Preferred skills
Master's degree in CS/Engineering/IT
Technologies
Amazon Redshift, PostgreSQL, MySQL, SQL Server, Hadoop, Spark, Kafka
Responsibilities
Design and develop scalable data pipelines for ingesting and processing large volumes of data; Implement and manage ETL processes to convert raw data into analytics-ready formats; Integrate data from diverse sources including databases, APIs, and third-party systems; Design, manage, and optimize databases, data warehouses, and data lakes; Write and optimize complex SQL queries; Implement data quality checks, validation rules, and monitoring mechanisms; Troubleshoot and debug data pipeline issues; Document data pipelines, workflows, and technical processes; Establish and enforce data engineering best practices including coding standards, version control, and testing
