Lead Software Engineer - Metrics & Payments Observability
Core
Design, build, and maintain scalable data pipelines, ETL/ELT workflows, and data platform components (catalogs, quality frameworks, metrics layers) for batch and real-time processing within a financial institution.
Role type
Lead Software Engineer (Data Platform & Observability)
Builds
Scalable data pipelines, ETL/ELT workflows, data catalogs, data quality frameworks, and semantic/metrics layers.
Domain
Financial Services / Data Engineering & Observability
Required skills
Python, Java, SQL, Apache Spark, Apache Flink, REST/GraphQL API design, Apache Airflow, AWS cloud services, AI-assisted development validation, Agile methodologies, CI/CD, application resiliency, security.
Preferred skills
Agentic AI, LLMs, RAG architectures, vector databases, embedding-based retrieval, Infrastructure as Code (Terraform), Docker, Kubernetes, data observability tools, BI platforms (Tableau, Sigma), dbt, MetricFlow, data modeling (star/snowflake schemas), ClickHouse, AWS data services (S3, Glue, Redshift, Athena, EMR, Lake Formation).
Technologies
Apache Spark, Apache Flink, Apache Airflow, AWS, Redshift, ClickHouse, Databricks, Docker, Kubernetes, Terraform, dbt, MetricFlow, Kafka, Spring Boot, GraphQL, Tableau, Sigma, Snowflake, Jira, Confluence
Responsibilities
Design and build scalable data pipelines and ETL/ELT workflows for batch and real-time processing; develop data platform components including data catalogs and quality frameworks; implement data modeling strategies to support analytics and reporting; establish engineering standards for Spring Boot services; ensure production readiness through observability and incident response; lead adoption of AI-assisted engineering practices; evaluate external vendors and startups for architectural suitability.
Seniority
Lead, hands-on IC with team leadership
