[Expression of Interest] Data Engineer
Core
Design, build, test, deploy, and operate scalable batch and streaming data pipelines for Experian's Ascend Sandbox cloud-based analytical platform serving internal teams and external clients.
Role type
Senior Data Engineer (Cloud/AWS)
Builds
Scalable data pipelines, trusted datasets/data products, and AWS-based data solutions for analytics and data science.
Domain
Financial services data infrastructure and cloud engineering
Deliverable
production ML models
Required skills
Python, PySpark, SQL, AWS (S3, Glue, Athena, Redshift, EMR), Data Modeling, ETL/ELT, Pipeline Orchestration, Data Quality/Observability, Data Governance, Spark Performance Optimization, CI/CD, Infrastructure as Code
Preferred skills
Lambda, Kinesis/MSK, DMS, Lake Formation, Terraform/CloudFormation/CDK, Git workflows, Jira/Bitbucket/Jenkins/Checkmarx, AI coding tools
Technologies
Python, PySpark, SQL, AWS, Airflow/MWAA, AWS Step Functions, AWS Glue Workflows, Terraform, AWS CloudFormation, AWS CDK, Parquet, JSON, CSV, Avro
Responsibilities
Design and optimize scalable ETL/ELT patterns with incremental processing and failure recovery; Engineer and optimize AWS-based data solutions; Implement automated data quality controls and monitoring; Collaborate with clients and data scientists to translate requirements into solutions; Investigate and resolve production data issues; Ensure compliance with architecture, security, and quality standards.
Seniority
Senior, hands-on IC
