Software Engineering Associate Advisor
Core
Design, develop, and maintain ETL processes to integrate data from Teradata into Databricks on AWS cloud, optimizing data pipelines for the corporate data warehouse.
Role type
Senior Data Engineering Associate
Builds
Data pipelines and ETL workflows for corporate data warehouse
Domain
Healthcare data engineering, cloud data infrastructure
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Teradata database development, SQL, Databricks, Apache Spark, AWS services (S3, Glue, Lambda, Airflow, EC2), Python, data warehousing concepts, CI/CD tools
Preferred skills
Agile methodology, capacity planning, security management
Technologies
Teradata, Databricks, AWS, Informatica, GitHub, Jenkins
Responsibilities
Design and maintain ETL processes, optimize Teradata queries and database structures, create data mappings and workflows in Informatica, monitor and troubleshoot data pipelines, develop data validations and quality checks, document technical specifications, assist in capacity planning and performance tuning
Seniority
Senior, hands-on IC