Senior Data Engineer - Microsoft Fabric
Core
Design, develop, test, and maintain high-quality data pipelines in Microsoft Fabric to drive platform delivery and solve sophisticated business problems.
Role type
Senior IC data engineer (cloud data platform)
Builds
Data pipelines, data lakes, and data warehousing solutions for analytics and modeling
Domain
Financial services / Cloud data engineering
Deliverable
production ML models
Required skills
Python, SQL, Scala, Pyspark/Apache Spark, Microsoft Fabric, AWS, DevOps automation, SQL/NoSQL database management, real-time analytics (Kinesis, Kafka), workflow orchestration (Airflow, AppFlow, Autosys, Cloudwatch, Splunk), data visualization (Tableau, Power BI), data protection and security, Infrastructure as Code (Ansible, CloudFormation), testing and quality assurance
Preferred skills
Serverless data pipeline development (AWS Glue, Lambda, Step functions)
Technologies
Microsoft Fabric, AWS, Python, SQL, Scala, Pyspark, Apache Spark, Kinesis, Kafka, Airflow, AppFlow, Autosys, Cloudwatch, Splunk, Tableau, Power BI, Ansible, CloudFormation
Responsibilities
Build and optimize data pipelines in Microsoft Fabric; develop high-quality, documented code adhering to standards; conduct complex data analysis and prepare data for modeling; collaborate with analysts and scientists on data projects; write unit, integration, and functional automation tests; coach others and embed learning in day-to-day work; analyze complex situations requiring in-depth evaluation of data factors.
Seniority
Senior, hands-on IC
