Senior Data Engineer, Mi-Platform
Core
Design, build, and launch robust end-to-end business solutions including data pipelines, products, and tools for the Market Intelligence team to uncover insights about markets and the broader economy.
Role type
Senior Data Engineer (AI-native cloud infrastructure)
Builds
Data pipelines, AI-ready data assets, reusable software components, libraries, and APIs
Domain
Financial services / Alternative data / AI
Deliverable
production ML models
Required skills
Python, distributed data processing (Spark), system design, scalable API development, cloud platforms (AWS/GCP/Azure), orchestration (Airflow)
Preferred skills
langraph/langchain, embedding models, vector databases, graph databases
Technologies
Spark, Airflow, AWS, Google Cloud Platform, Azure, Python
Responsibilities
Design and develop big data processing pipelines for ingestion, transformation, and trends creation; Develop reusable software components and APIs; Implement data quality, lineage, and governance controls; Build scalable AI-native cloud-based infrastructure; Optimize performance, cost, and reliability of software systems
