Data Engineer
Core
Building and supporting data-driven applications, data lakes, and data warehouses to enable reporting analytics and innovation.
Role type
Data Engineer
Builds
Modular and scalable data ELT/ETL pipelines and data infrastructure
Domain
Enterprise data infrastructure and analytics
Deliverable
production ML models | product features | infrastructure
Required skills
Python, SQL, Spark, Azure Data Factory, Azure Data Lake, Azure SQL Databases, Azure Data Warehouse, Azure Synapse Analytics, Azure Databricks, Power BI, dimensional data modeling, data ingestion, data curation, pipeline optimization, metadata management
Preferred skills
Agile/SCRUM framework experience, cross-functional collaboration
Technologies
Python, Spark, SQL, Azure Data Factory, Azure Data Lake, Azure SQL Databases, Azure Data Warehouse, Azure Synapse Analytics, Azure Databricks, Microsoft Purview, Power BI
Responsibilities
Build and productionize modular and scalable data ELT/ETL pipelines; Implement data ingestion and curation pipelines for business intelligence; Clean, prepare, and optimize datasets for performance; Support Business Intelligence Analysts in data modeling for visualization; Provide production support for pipeline issues and data integrity; Collaborate with architects and analysts to develop data pipelines; Identify and implement internal process improvements for automation and scalability; Assist in building and maintaining data catalogs and metadata; Develop optimized data pipelines using Python, Spark, and SQL; Document as-built pipelines and utilize source control.
Seniority
Mid-level, hands-on IC