Lead Software Engineer - Big Data(Data Engineer), Python, and AWS
Core
Lead Data Engineer designing, building, and maintaining scalable data pipelines, architectures, and data products on AWS to support business objectives.
Role type
Lead Software Engineer (Data Engineer)
Builds
Data collection, storage, access, and analytics platforms; semantic layers using Iceberg tables; automated data workflows.
Domain
Financial Services / Big Data Engineering / Cloud
Deliverable
production ML models | product features
Required skills
Python, Apache Spark, AWS (S3, Step Functions, EMR, Glue, Lambda, EventBridge, API Gateway), Databricks, Terraform, SQL, NoSQL, Data Modeling, CI/CD, Infrastructure as Code
Preferred skills
Real-time data processing, AI/ML-oriented data use cases, Snowflake, Data Engineering certifications
Technologies
AWS, Apache Spark, Databricks, Iceberg, Terraform, Redshift, RDS, Snowflake
Responsibilities
Design and implement scalable data pipelines and data matching solutions; develop secure production code and review/debug others' code; troubleshoot application code and data workflows; evaluate access control processes for data asset security; automate remediation of recurring issues.
Seniority
Lead, hands-on IC with mentorship
