Data & AI Engineer
Core
Design, build, and optimize scalable ETL/ELT pipelines and Lakehouse architectures using Apache Spark and Delta Lake to support production AI workloads.
Role type
Senior IC Data & AI Engineer (Lakehouse & MLOps)
Builds
Cloud-agnostic Data, ML, and AI solutions for client founders and tech leaders
Domain
Big Data, Machine Learning, Cloud Infrastructure
Deliverable
production ML models
Required skills
Apache Spark, Delta Lake, Python, SQL, MLflow, Terraform, CI/CD, Infrastructure-as-Code
Preferred skills
Databricks Certified Data Engineer, Databricks ML Professional certification
Technologies
Databricks, AWS, Azure, GCP, Unity Catalog, MLflow, Terraform
Responsibilities
Design and optimize scalable ETL/ELT pipelines using Apache Spark and Delta Lake; Fine-tune Spark jobs for low latency and high throughput; Implement CI/CD pipelines and Infrastructure-as-Code for automated deployments; Enforce enterprise data security and compliance standards within Unity Catalog.
Seniority
Senior, hands-on IC