Senior Data Engineer, Unified Platform

## Responsibilities - Model, build, and manage data products built atop DRW's Unified Data Platform. - Work closely with Data Strategists to determine appropriate data sources and implement processes that onboard and manage new datasets for trading, research, and back-office purposes. - Design data governance processes that enable discovery, cost-sharing, usage tracking, access controls, and quality control of datasets to address the needs of DRW trading teams and strategies. - Continually monitor data ingestion pipelines and data quality to ensure stability, reliability, and quality of the data. Design monitoring and quality control software and processes. - Own the technical aspects of vendor ingestion pipelines, coordinating with vendor relationship managers on upcoming changes, performing routine data operations without breaking internal users, and contributing to the team's on-call rotation to respond to unanticipated changes. - Rapidly respond to user requests, identifying platform gaps, and self-service opportunities that make the user experience more efficient. ## Requirements - 7+ years of experience working with modern data technologies and building data-first products. - Excellent written and verbal communication skills. - Proven ability to work in a collaborative, agile, and fast-paced environment, prioritizing multiple tasks and projects, and efficiently handle the demands of a trading environment. - Proven ability to deliver rapid results within processes that span multiple stakeholders. - Strong technical problem-solving skills. - Extensive familiarity with SQL and Java or Python, with a proven ability to develop and deliver maintainable data transformations for production data pipelines. - Experience leveraging data modeling techniques and ability to articulate the trade-offs of different approaches. - Experience with one or more data processing technologies (e.g. Flink, Spark, Polars, Dask, etc.). - Experience with multiple data storage technologies (e.g. S3, RDBMS, NoSQL, Delta/Iceberg, Cassandra, Clickhouse, Kafka, etc.) and knowledge of their associated trade-offs. - Experience with multiple data formats and serialization systems (e.g. Arrow, Parquet, Protobuf/gRPC, Avro, Thrift, JSON, etc.). - Experience managing data pipeline orchestration systems (e.g. Kubernetes, Argo Workflows, Airflow, Prefect, Dagster, etc.), delving deeply into low-level details. - Proven experience in managing the operational aspects of large data pipelines such as backfilling datasets, rerunning batch jobs, monitoring event throughput, and handling dead-letter queues. - Prior experience triaging data quality control processes, correcting data gaps and inaccuracies. ## Nice to Have - None specified. ## Benefits - The annual base salary range for this position is $200,000 to $250,000 depending on the candidate's experience, qualifications, and relevant skill set. - The position is also eligible for an annual discretionary bonus. - DRW offers a comprehensive suite of employee benefits including group medical, pharmacy, dental and vision insurance, 401k (with discretionary employer match), short and long-term disability, life and AD&D insurance, health savings accounts, and flexible spending accounts. - For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at https://drw.com/privacy-notice. - California residents, please review the California Privacy Notice for information about certain legal rights at https://drw.com/california-privacy-notice.