Data Engineer
Core
Design, build, and maintain large-scale data pipelines and platforms to transform raw data into actionable insights for decision-makers.
Role type
Data Engineer
Builds
Data warehouses, data lakes, operational data stores, and scalable data pipelines
Domain
Cloud data engineering and big data
Required skills
SQL, Python, ETL/ELT pipeline design, database design, cloud platforms (AWS, Azure, Google Cloud), big data frameworks (Spark, Kafka, Hadoop), web scraping, data modeling, REST APIs
Preferred skills
Databricks, Data Fabric (Microsoft), Spring, AWS Lambda/ECS/Eventbridge/Glue, data governance
Technologies
SQL, Python, R, SSIS, AWS DMS, AWS Lambda, ECS, Eventbridge, AWS Glue, Spring, SQL Server, PostgreSQL, MongoDB, MySQL, SQLite, VoltDB, Cassandra, AWS S3, Athena, Hadoop, Spark, Kafka, RabbitMQ, BeautifulSoup, CasperJS, PhantomJS, Selenium, Nodejs
Responsibilities
Design and deploy data tables, views, and marts; build and maintain batch and real-time data pipelines; integrate and collate data silos; develop backend APIs and databases; collaborate with cross-functional teams in an Agile environment
Seniority
Mid-level to Senior, hands-on IC
