Senior Data Engineer
Core
Designing and building a new data platform to manage over 100 million objects with high throughput, scalability, data quality, lineage, and auditability.
Role type
Senior IC data engineer (cloud infrastructure & pipelines)
Builds
Metadata-driven, event-based distributed data processing platform and low-latency APIs
Domain
Financial data / Credit intelligence / Cloud infrastructure
Deliverable
production ML models
Required skills
Python, Spark, Airflow, AWS (S3, Iceberg, Aurora, DynamoDB, Lambda, Fargate, ECS, Glue), Terraform, CI/CD, distributed systems design, batch and streaming processing, API development, code review, mentorship
Preferred skills
AI tools usage (Perplexity, Cursor, CoPilot), domain data understanding, whiteboarding, collaborative decision sharing
Technologies
Python, Spark, Airflow, Iceberg, DynamoDB, AWS Glue, S3, Terraform, API Gateway, Athena, Redshift, Lambda
Responsibilities
Design and build metadata driven, event based distributed data processing platform; Build and scale data pipelines combining batch and streaming approaches; Develop highly available and low latency APIs; Contribute to codebase and participate in peer reviews; Mentor engineers and provide leadership on complex technology issues; Take ownership of end-to-end deliverables from planning to production.
Seniority
Senior, hands-on IC with mentorship
