Senior Data Engineer
Core
Lead the modernization of a large-scale, cloud-based data platform, migrating to a Lakehouse architecture and establishing data quality and governance practices.
Role type
Senior Data Engineer (Infrastructure & Platform)
Builds
Production-grade data infrastructure, Lakehouse architecture, automated data quality frameworks, and secure data access patterns for internal tools and AI agents.
Domain
Commercial Real Estate / Data Engineering / Cloud Infrastructure
Deliverable
production ML models | infrastructure
Required skills
End-to-end ownership of production pipelines, Spark and distributed processing, Expert-level SQL, Advanced Python (factory patterns, dynamic DAG generation), Complex batch pipeline migration, Data quality framework implementation
Preferred skills
Web scraping or API ingestion at scale, LLM/agent approaches for data extraction, MCP (Model Context Protocol) familiarity, Databricks expertise (jobs/workflows, catalog/governance, Delta)
Technologies
Databricks, Spark, SQL, Python, Delta
Responsibilities
Lead migration from current warehouse to Databricks Lakehouse, Redesign pipeline architecture for efficiency, Design and implement robust data quality frameworks, Architect secure data access protocols for internal users and AI agents, Create comprehensive documentation for migration and operations, Partner with domain experts to translate business requirements into automated validation
Seniority
Senior, hands-on IC
