Principal Data Engineer -General application
Required skills
5 + years of experience in Data Engineer role, Degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field, Strong command of English, AWS (Batch, CloudFormation, IAM, Lambda, ECS, StepFunctions), Terraform, Python, SQL, Relational databases (e.g., PostgreSQL), API (RESTful), BI Tools (QuickSight)
Preferred skills
Familiarity with Agile and SCRUM methodologies, Experience working with PowerBI to develop dashboards, Analytical skills related to working with unstructured datasets, A successful history of processing value from large, disconnected datasets, Experience working with agile, globally distributed teams, Design and develop prompts for various applications, including text generation, translation, question answering, and creative writing, Collaborate with product teams, data scientists, and engineers to understand user needs and translate them into effective prompts
Technologies
AWS, Terraform, Python, SQL, PostgreSQL, RESTful API, QuickSight
Responsibilities
Create data pipeline architecture, Assemble complex data sets that meet functional / non-functional requirements, Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, Build the infrastructure required for extraction, transformation, and loading of data from different data sources using SQL and AWS 'big data' technologies, Build analytics tools that use the data pipeline to provide applicable insights into employee experience, operational efficiency and other main performance metrics, Work with team members to assist with data-related technical issues and support associated data infrastructure needs, Build processes supporting data transformation, data structures, metadata, dependency and workload management, Keep up to date with the latest and greatest in feature-sets and capabilities from public cloud providers (such as AWS) and find ways to apply them back help their team, Work with data scientists and analysts to work toward greater functionality in our data systems
Seniority
Principal
Domain
Data Engineering, Enterprise Software, Customer Success
Full job description
NOTE: This is NOT an open position. Please submit your CV here for future consideration.
Principal data engineer
Position Overview
We are looking for an experienced Principal Data Engineer to join our growing Global Delivery Team within the Enterprise Customer Success Department. You will improve our data and data pipeline architecture and optimizing data flow and collection for associated teams. You are an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will support software developers, data analysts and data scientists on data projects and will ensure data delivery architecture is consistent throughout ongoing projects.
We support hybrid working schedule.
Responsibilities
- Create data pipeline architecture
- Assemble complex data sets that meet functional / non-functional requirements
- Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
- Build the infrastructure required for extraction, transformation, and loading of data from different data sources using SQL and AWS 'big data' technologies
- Build analytics tools that use the data pipeline to provide applicable insights into employee experience, operational efficiency and other main performance metrics
- Work with team members to assist with data-related technical issues and support associated data infrastructure needs
- Build processes supporting data transformation, data structures, metadata, dependency and workload management
- Keep up to date with the latest and greatest in feature-sets and capabilities from public cloud providers (such as AWS) and find ways to apply them back help their team
- Work with data scientists and analysts to work toward greater functionality in our data systems
Minimum Qualifications
- 5 + years of experience in Data Engineer role
- Degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
- Strong command of English
- Technical Skills
- AWS (Batch, CloudFormation, IAM, Lambda, ECS, StepFunctions)
- Terraform
- Python
- SQL
- Relational databases (e.g., PostgreSQL)
- API (RESTful)
- BI Tools (QuickSight)
Preferred Qualifications
- Familiarity with Agile and SCRUM methodologies
- Experience working with PowerBI to develop dashboards
- Analytical skills related to working with unstructured datasets
- A successful history of processing value from large, disconnected datasets
- Experience working with agile, globally distributed teams
- Design and develop prompts for various applications, including text generation, translation, question answering, and creative writing
- Collaborate with product teams, data scientists, and engineers to understand user needs and translate them into effective prompts
- Analyse and iterate on performance metrics and user feedback to
