Data Engineer - Capital Markets
Core
Design and maintain enterprise data Lakehouse infrastructure to support real-time financial trading data consumption and advanced analytics.
Role type
Senior Databricks Engineer (Capital Markets)
Builds
High-performance data pipelines, regulatory reporting environments, and real-time data views for trading.
Domain
Financial Services / Capital Markets / Cloud Data Engineering
Deliverable
production ML models
Required skills
Databricks (Medallion Architecture, DLT, Unity Catalog), AWS (S3, Glue, IAM, MSK/Kafka), Python (PySpark), SQL, Apache Spark internals, Delta Lake, CI/CD (Terraform, DABs, GitHub Actions)
Preferred skills
Databricks Certified Data Engineer Professional, AWS Certified Data Engineer - Professional
Technologies
Databricks, AWS, PySpark, Delta Lake, Terraform, Unity Catalog, Kafka, MSK
Responsibilities
Design and implement robust data pipelines using Databricks Medallion Architecture; Develop, scale, and orchestrate complex data workflows utilizing Databricks Jobs and Delta Live Tables; Standardize and automate deployment of Databricks assets and code pipelines; Ensure seamless integration between Databricks and AWS services (S3, Glue, IAM); Tune Spark clusters and optimize Delta Lake storage; Implement fine-grained data access controls and data lineage using Unity Catalog.
Seniority
Senior, hands-on IC