Principal Data Engineer
Core
Design and develop scalable data ingestion and ETL/ELT pipelines for structured, semi-structured, and unstructured data to support application, search, analytics, and AI-enabled use cases.
Role type
Senior IC data engineer (enterprise data platforms)
Builds
Scalable data platforms, ETL/ELT pipelines, and data-processing workflows
Domain
Enterprise IT, Wealth & Asset Management, AI-enabled data solutions
Deliverable
production ML models
Required skills
Python, SQL, PostgreSQL, Amazon RDS, DynamoDB, Amazon S3, ETL/ELT pipeline design, data modeling, data quality, metadata, governance, security, operational monitoring, document processing, vector-based retrieval (pgvector)
Preferred skills
AWS-native data services, serverless patterns, Agile/Scrum, CI/CD
Technologies
AWS, CI/CD, DevOps, JSON, Parquet, pgvector, Amazon Textract
Responsibilities
Design and develop scalable data ingestion and ETL/ELT pipelines; Create reusable data-processing and transformation components; Design and support data models; Support document ingestion and extraction using Amazon Textract; Prepare and transform data for application, search, analytics, and AI-enabled use cases; Implement controls for data quality, validation, reconciliation, metadata, lineage, retention, and governance; Provide secure access to data and implement encryption; Optimize pipelines and databases for performance, reliability, scalability, and cost; Monitor data processes and troubleshoot processing and integration issues.
