Staff Data Engineer (Databricks specialist)
Core
Lead the design and development of scalable, enterprise-grade data platforms with a strong specialization in Databricks and modern cloud technologies, driving complex cross-functional data initiatives and AI-powered data capabilities.
Role type
Staff Data Engineer (Databricks Specialist)
Builds
Enterprise data platforms, MDM solutions, agentic AI/LLM-powered solutions, feature stores, vector databases, and backend data services/APIs.
Domain
Enterprise data engineering, Master Data Management (MDM), AI/LLM integration, Cloud-native architecture
Deliverable
production ML models
Required skills
Enterprise-scale data platform architecture, Databricks, SQL, Python, Cloud-native distributed systems, MDM (entity resolution, match-and-merge), Agentic AI/LLM orchestration, REST/GraphQL APIs, Data governance and lineage, CI/CD and DevOps practices
Preferred skills
Scala or Java, Informatica/Reltio/Profisee, Unity Catalog/Microsoft Purview/Collibra, Event-driven architectures (Kafka, Flink), Containerization (Docker, Kubernetes), Workflow orchestration (Airflow, Dagster, dbt)
Technologies
Databricks, Azure Synapse, BigQuery, Amazon Redshift, Snowflake, Delta Lake, Apache Iceberg, Parquet, LangChain, AutoGen, Semantic Kernel, MCP, Kafka, Azure Event Hubs, Amazon Kinesis, Flink, Spark Structured Streaming, Docker, Kubernetes, Airflow, Dagster, dbt
Responsibilities
Own and drive technical architecture for complex data initiatives; Design and maintain scalable data pipelines using Databricks and cloud technologies; Lead enterprise MDM initiatives; Architect and integrate agentic AI and LLM-powered solutions; Build data foundations for ML/AI applications; Design and deliver backend data services and APIs; Establish engineering standards and data governance frameworks; Lead technical design reviews and mentor engineers; Collaborate with leadership to define data roadmaps; Resolve systemic performance and scalability challenges.
Seniority
Staff, strategic leadership with hands-on architecture and mentorship