Senior Data Engineer
Core
Design, build, and implement generalized large-scale data pipelines for Sport Performance products using Nifi, AWS, and Kafka.
Role type
Senior Data Engineer
Builds
Scalable data ingestion pipelines, ETL processes, and data models for downstream analytics and data science.
Domain
Sports technology, cloud data engineering
Deliverable
production ML models
Required skills
Python, Java, Apache NiFi, AWS, Apache Kafka, SQL, Docker, ETL development, data warehousing, data modeling, unit testing, observability, error handling, data cleaning, SQL debugging
Preferred skills
Agile methodologies, JIRA, self-driven independent work
Technologies
Apache NiFi, AWS, Apache Kafka, Docker, JIRA
Responsibilities
Design and develop scalable Nifi ingestion pipelines within AWS cloud services; Create data pipeline triggers and filters to optimize data flow and resource usage; Implement monitoring and error handling for data pipelines; Establish rigorous unit testing across the data pipeline; Design and create data models for the ETL system; Build automated cleaning of data to remove bad or unusable data; Conduct data analytics and debugging of bad data by writing SQL queries; Collaborate with other engineering teams to adopt standard methodologies and drive scalability.
Seniority
Senior, hands-on IC
