Senior Data Platform Engineer - Python & AWS
Core
Build and scale real-time data ingestion, processing, storage, and delivery systems for financial market intelligence.
Role type
Senior IC data platform engineer (Python/AWS)
Builds
Production data pipelines, event-driven systems, and AWS infrastructure for news and market data.
Domain
Financial technology / Data engineering
Deliverable
production ML models
Required skills
Python, AWS (S3, RDS/PostgreSQL, Lambda, ECS, EKS), SQL, Infrastructure-as-Code (Pulumi/Terraform), CI/CD, Linux, Containers, Distributed systems, Event-driven architecture
Preferred skills
Apache Airflow, Kubernetes, Real-time data processing, AI/ML infrastructure, LLM workflows, GPU infrastructure, Financial-market data
Technologies
Python, AWS, S3, RDS, PostgreSQL, Lambda, ECS, EKS, Pulumi, GitHub Actions, Linux (via careerplan.io/jobs/7ca1d033-37d9-452d-8871-c02093621537-senior-data-platform-engineer-python-aws-at-permutable)
Responsibilities
Design and develop Python services and pipelines to ingest, validate, transform, and deliver large volumes of data; Build and scale AWS infrastructure; Develop event-driven systems with messaging and asynchronous processing; Improve data quality with validation, deduplication, and recovery mechanisms; Optimize storage and data access for research and client applications; Support AI and quantitative workloads by integrating model outputs; Automate infrastructure and deployments; Monitor and optimize production pipeline health and performance.
Seniority
Senior, hands-on IC
