Data Engineer
Core
Build Python-based data validation and comparison frameworks to ensure the integrity, accuracy, and consistency of critical business data for financial services clients.
Role type
Senior IC data engineer (data quality & validation)
Builds
Automated data validation workflows, reconciliation frameworks, and quality check scripts
Domain
Financial services / Data engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python, Pandas, NumPy, SQL, ETL/Data Pipeline fundamentals, Data cleansing and transformation, Root cause analysis
Preferred skills
PySpark, Azure Data Factory, Databricks, Power BI, Git, CI/CD, Cloud platforms (Azure, AWS, GCP)
Responsibilities
Validate and reconcile multiple datasets simultaneously; Build Python-based data validation and comparison frameworks; Use Pandas and NumPy to analyze, transform, and validate large datasets; Develop automated processes to identify missing, duplicate, or inconsistent data; Investigate data quality issues and determine root causes; Create reports highlighting discrepancies and data quality metrics
Seniority
Mid-Senior, hands-on IC