Lead AWS Software Data Engineer
Core
Designing and developing scalable data pipelines and lakehouse architectures to power financial indices trusted by global institutions.
Role type
Lead Software Data Engineer
Builds
Scalable data pipelines, lakehouse architecture (Apache Iceberg), and robust data workflows for financial indices.
Domain
Financial markets infrastructure / Data Engineering
Deliverable
production ML models | product features
Required skills
Python (clean, testable, type hints), Apache Spark, AWS data stack (Glue, EMR, Lambda, S3), Relational databases (PostgreSQL), ETL pipeline design, CI/CD, Automated testing, Observability implementation
Preferred skills
Apache Iceberg, Data quality frameworks (Great Expectations, Deequ), Databricks, Financial markets domain knowledge
Technologies
Python, Apache Spark, AWS Glue, AWS EMR, AWS Lambda, AWS S3, Apache Iceberg, PostgreSQL, GitLab CI, Jenkins, GitHub Actions
Responsibilities
Designing and developing scalable, testable data pipelines; Orchestrating data workflows with AWS tools; Contributing to lakehouse architecture development; Building observability and quality checks into data flows; Collaborating with business teams to translate requirements into solutions; Participating in code reviews and architecture discussions.
Seniority
Lead, hands-on IC with mentorship responsibilities