Staff Data Engineer (Databricks specialist)
Core
Lead the design and development of scalable, enterprise-grade data platforms with a strong specialization in Databricks, driving complex cross-functional data initiatives, MDM, and AI-powered data capabilities.
Role type
Staff Data Engineer (Databricks Specialist)
Builds
Enterprise data platforms, MDM solutions, agentic AI/LLM pipelines, feature stores, and backend data services
Domain
Enterprise Data Engineering, Master Data Management, AI/LLM, Cloud Data Platforms
Deliverable
production ML models
Required skills
Databricks, Python, SQL, Cloud-native architecture, Distributed systems, Data modeling, ETL/ELT, MDM (entity resolution, match-and-merge), Agentic AI/LLM orchestration, REST/GraphQL APIs, Relational/NoSQL databases, Delta Lake/Iceberg, CI/CD, IaC, DevOps/DataOps
Preferred skills
Scala/Java, Snowflake/Azure Synapse/BigQuery/Redshift, Data governance tools (Unity Catalog, Purview, Collibra), Event-driven architectures (Kafka, Flink), Containerization (Docker, Kubernetes), Workflow orchestration (Airflow, Dagster, dbt), Commercial real estate/fintech domain experience
Technologies
Databricks, Azure, AWS, GCP, Snowflake, Azure Synapse, BigQuery, Amazon Redshift, Delta Lake, Apache Iceberg, Parquet, LangChain, AutoGen, Semantic Kernel, MCP, Kafka, Azure Event Hubs, Amazon Kinesis, Flink, Spark Structured Streaming, Docker, Kubernetes, Airflow, Dagster, dbt, Unity Catalog, Microsoft Purview, Collibra, Alation
Responsibilities
Own and drive technical architecture for complex data initiatives; Design and maintain scalable data pipelines using Databricks and cloud technologies; Lead enterprise MDM initiatives including golden records and entity resolution; Architect and integrate agentic AI and LLM-powered solutions; Build data foundations for ML/AI including feature stores and vector databases; Design and deliver backend data services and APIs; Establish engineering standards for data modeling, quality, and CI/CD; Implement enterprise-wide data governance and lineage frameworks; Lead technical design reviews and mentor senior engineers; Collaborate with leadership to define scalable data roadmaps; Resolve systemic performance and scalability challenges.
Seniority
Staff, strategic leadership with hands-on architecture and mentorship