Data Engineer
Core
Design, develop, and troubleshoot scalable enterprise-level data pipelines for real-time and batch processing, operationalizing ML models, and ensuring data quality.
Role type
Senior Product Software Data Engineer
Builds
Real-time and batch data pipelines, data governance frameworks, and validation mechanisms
Domain
Enterprise data engineering, cloud data platforms
Deliverable
production ML models | data pipelines | dashboards & analysis
Required skills
Data pipeline design, Big data technologies, Relational data modeling, T-SQL, Azure ecosystem, Data governance, Code review, Mentoring
Preferred skills
Python, Scala, Power BI, Azure Analysis Services, Agile/Scrum, CI/CD
Technologies
Scala, Apache Spark, Azure Databricks, PySpark, Hadoop, Kafka, EventHub, Stream Analytics, SQL Server, Azure Data Factory, Azure Data Lake, Azure SQL DB, Azure Synapse Analytics, Azure ML, Git
Responsibilities
Design and develop optimized, reliable, and secure enterprise-level data pipelines for real-time and batch processing; Analyze existing business processes to recommend data-driven improvements; Partner with data scientists to operationalize machine learning models; Champion data governance, metadata management, and operational excellence; Ensure data quality and integrity by developing validation frameworks and monitoring mechanisms; Mentor junior engineers and contribute to knowledge sharing.
Seniority
Senior, hands-on IC with mentorship responsibilities
