Senior Software Engineer - Python, Data Engineering, AI
Core
Design, build, and operate scalable batch and incremental ETL/ELT pipelines, manage cloud data lakehouses, and expose datasets to AI/ML layers for the beauty and wellness industry.
Role type
Senior Data Engineer (Python, Cloud, Lakehouse)
Builds
Production-grade data pipelines, curated lakehouse tables (Iceberg/Delta), and governed datasets for analytics and AI agents.
Domain
Cloud Data Engineering, Lakehouse Architecture, Beauty & Wellness SaaS
Deliverable
production ML models
Required skills
Python (3.x, pandas, PySpark), Apache Spark/PySpark at scale, Advanced SQL (Trino/Athena), Lakehouse fundamentals (Iceberg/Delta Lake), Cloud data platforms (AWS/Azure), DAG orchestration, Incremental ingestion, CI/CD, Data quality & observability
Preferred skills
Databricks (Delta Live Tables, Unity Catalog), Azure data stack, Infrastructure as code (Terraform/CloudFormation), Streaming (Kafka/Kinesis), Relational extraction, BI serving, AI/LLM data surfaces, Statistics for data quality
Technologies
Python, PySpark, AWS (Glue, Athena, S3, Lambda, Step Functions), Azure (ADLS, Data Factory, Synapse), Databricks, Trino, Delta Lake, Apache Iceberg, Terraform, CloudFormation, Docker, QuickSight, Kafka, Kinesis
Responsibilities
Design and operate ETL/ELT pipelines; Model and tune Iceberg/Delta Lake tables; Build ingestion from external APIs; Extend data-quality frameworks; Lead Databricks-on-Azure lakehouse workstreams; Ship code via PRs with tests and IaC; Partner with analysts on metric definitions; Mentor junior engineers
Seniority
Senior, hands-on IC
