Senior Data Engineer - Asian Timezones
Core
Architecting, implementing, and operating the complete data infrastructure pipeline for Research and Trading operations, handling vast amounts of market trading data, real-time news feeds, and internal/external data sources.
Role type
Senior Data Engineer (Financial Services/Proprietary Trading)
Builds
Scalable, reliable, and cost-efficient data pipelines and data warehouse infrastructure for trading and research teams.
Domain
Financial Services / Proprietary Trading / Data Engineering
Deliverable
production ML models | infrastructure
Required skills
Complex SQL, Python or Java/Scala, ETL pipeline design (Apache Airflow, dbt), Dimensional modeling (Star/Snowflake schemas), Big Data processing (Apache Spark, Kafka, Flink), Cloud data platforms (Snowflake, Redshift, BigQuery, Synapse), Data quality checks and anomaly detection, Data security and privacy standards.
Preferred skills
Cloud platform certifications (AWS, GCP, Azure, Snowflake), Data quality frameworks (Great Expectations, Deequ), Infrastructure-as-code (Terraform, CloudFormation), Containerization (Docker, Kubernetes), Observability tools (Monte Carlo, Datadog), Machine learning pipelines, Prior experience in trading or financial services.
Technologies
Apache Airflow, dbt, Apache Spark, Kafka, Flink, Snowflake, Redshift, BigQuery, Synapse, AWS, GCP, Azure, Terraform, CloudFormation, Docker, Kubernetes, Monte Carlo, Datadog, Git/GitHub.
Responsibilities
Design and optimize scalable pipelines for ingesting and transforming large-volume datasets; Ensure data quality, consistency, and real-time monitoring; Develop processes to normalize and organize the data warehouse; Apply advanced data management practices for scalability and efficiency; Collaborate with research and analytics teams to build frameworks for data exploration and model development; Optimize data storage, processing, and management for cost-effectiveness.
Seniority
Senior, hands-on IC
