Senior, Data Engineer
Core
Design and maintain AI-enabled, PySpark data pipelines on AWS to process large datasets at scale, build internal dashboards, and enforce data quality.
Role type
Senior, hands-on IC data engineer
Builds
Production data pipelines, analytics-ready fact/dimension tables, internal dashboards, and data products
Domain
Cloud data engineering, big data processing, AI/ML integration
Deliverable
production ML models
Required skills
PySpark, SQL, AWS EMR, AWS Glue, AWS Athena, AWS S3, AWS DynamoDB, CloudFormation, CI/CD, observability, data quality testing, GenAI/LLM integration
Preferred skills
None stated
Technologies
PySpark, Greenplum, Postgres, AWS EMR, AWS Glue, AWS Athena, AWS S3, AWS DynamoDB, AWS Amplify, AWS Bedrock, CloudFormation, CodeCommit, CodeBuild, CodePipeline, CloudWatch
Responsibilities
Develop and execute software tests to identify software problems; Write and optimize complex SQL Scripts and build Python scripts to execute large-scale transformations on Greenplum; Design and maintain AI-enabled, PySpark data pipelines on AWS primarily EMR, Glue, Athena, DynamoDB, and S3; Harden and tune jobs for performance, cost, and compliance; Build internal dashboards and data products and instrument monitoring/alerting
Seniority
Senior, hands-on IC
