Big Data Engineer
Core
Architect and maintain large-scale, efficient data pipeline architectures and optimize data systems for the Enterprise Data and Advanced Analytics Platform.
Role type
Senior Big Data Engineer
Builds
Robust data pipelines, ETL infrastructure, and optimized data delivery systems for capital markets.
Domain
Financial services / Capital markets / Cloud data engineering
Deliverable
production ML models | product features | infrastructure
Required skills
Big Data technologies (Spark, Hive, Presto), AWS cloud services (S3, EC2, EMR), Snowflake, Python/Scala/Java, SQL, Data modeling, Performance tuning, ETL development, Cloud Infrastructure management
Preferred skills
Kafka, Terraform, Ansible, Equity Markets domain knowledge
Technologies
AWS (S3, EC2, EMR, VPC, CloudFormation, CloudWatch, Managed Airflow, SNS), Snowflake, Spark, Hive, Presto, Kafka, Terraform, Ansible, GIT
Responsibilities
Design and manage large-scale data pipeline architectures; Automate manual tasks and optimize data delivery infrastructure; Administer AWS cloud services and manage data ingestion/ETL operations; Provide on-call operational support for system reliability; Mentor junior staff and manage stakeholder expectations.
Seniority
Senior, hands-on IC with mentorship responsibilities
