Senior Data Engineer
Core
Architect high-performance data infrastructure and real-time pipelines enabling agentic AI to reason over complex enterprise environments.
Role type
Senior IC data engineer (AI/Enterprise Data)
Builds
Real-time data pipelines, ETL/ELT workflows, and data integration solutions for autonomous systems.
Domain
Enterprise technology, AI infrastructure, Cloud data engineering.
Deliverable
production ML models
Required skills
Azure data services, SQL, Python, ETL/ELT design, Data modeling, CI/CD pipelines, PySpark, REST APIs, Event Hubs, Azure Functions, Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Blob Storage, Azure Key Vault, Azure Monitor, Microsoft Entra ID, Apache Spark.
Preferred skills
Java.
Technologies
Microsoft Azure, PySpark, Apache Spark, Databricks, SQL, Python, Java.
Responsibilities
Design and develop scalable data pipelines using Azure services; Build and maintain ETL/ELT workflows for structured and unstructured data; Implement data integration solutions across multiple sources; Ensure data quality, security, governance, and compliance; Optimize data processing performance and cost efficiency; Develop reusable frameworks, standards, and best practices; Monitor and troubleshoot Azure-based data pipelines; Collaborate with cross-functional teams; Support analytics, reporting, and machine learning initiatives; Implement CI/CD pipelines and maintain documentation.
Seniority
Senior, hands-on IC.