Senior Data Engineer
Core
Senior Data Engineer leading the transition of data pipelines to an enterprise Databricks/AWS lakehouse platform, focusing on performance, scalability, and secure cloud-native patterns.
Role type
Senior IC data engineer (cloud data platform & lakehouse)
Builds
Cloud-native data pipelines, data lakehouse architectures, and asset-agnostic onboarding frameworks on AWS/Databricks
Domain
Financial services data engineering, cloud data platforms, and data mastering
Deliverable
production ML models | product features
Required skills
AWS services (S3, Glue, Lambda, Kinesis, Lake Formation, IAM), Databricks (Delta Lake, Unity Catalog), PySpark, SQL, CI/CD, Infrastructure as Code, Data Quality, Metadata management, Prompt engineering, AI-assisted code review
Preferred skills
Experience with NeoXam DataHub, Golden record generation, Offshore/global team collaboration
Technologies
AWS, Databricks, Delta Lake, Apache Iceberg, Python, SQL, AWS CodePipeline, AWS CodeBuild
Responsibilities
Migrate and optimize existing data pipelines to the enterprise platform; Implement repeatable engineering patterns for ingestion, transformation, and deployment; Apply AI-assisted tools for code generation, optimization, and documentation; Integrate pipelines with the data mastering platform for golden record generation; Establish cloud engineering standards and operational discipline for production pipelines
Seniority
Senior, hands-on IC