Senior AI Developer - Python and Spark
Core
Build and maintain a path from discovery to production for Python/PySpark models running on Databricks batch jobs and real-time Kafka streams.
Role type
Senior AI Developer (MLOps & Data Engineering)
Builds
Production ML models and event-driven data pipelines
Domain
Big Data, AI/ML, Cloud Infrastructure
Deliverable
production ML models
Required skills
Python, PySpark, Databricks, AWS, Kafka, MLOps, Java, Spring Boot, Terraform, Docker, MLflow
Preferred skills
Unity Catalog, Delta Lake, ECS/EKS, ALB, Dynatrace, GitLab CI/CD
Technologies
AWS, Databricks, Kafka, MLflow, Terraform, Docker, GitLab, Dynatrace, Spring Boot, Unity Catalog, Delta Lake
Responsibilities
Build and maintain paths for Python/PySpark models to run on Databricks batch and Kafka streams; Partner with Product Owners to turn architecture into sprint work; Improve MLOps practices including testing, model versioning, and drift monitoring; Support Data Science in evaluating algorithms and data feeds; Collaborate with engineers to provide code/design feedback; Explain technical trade-offs to business stakeholders.