HQ BARCELONA - Senior Data Engineer
Core
Design and implement data pipelines and architecture to support AI models, data storage, and processing for a workforce on-demand platform.
Role type
Senior Data Engineer (Data Architecture & Pipelines)
Builds
Data lake, data warehouse, ETL pipelines, and data storage services for AI training and product operations.
Domain
Workforce management / HR Tech / Big Data
Deliverable
production ML models
Required skills
Python, Java/Scala, Big Data technologies (Hadoop, Spark, Hudi, Flink), ETL workflow management, data architecture design, streaming platforms (Kafka, Kinesis, RabbitMQ), data storage formats (Parquet, Avro, JSON), production-grade coding practices.
Preferred skills
Open source contributions, technical writing, knowledge of 3rd party integrations (Salesforce, Kustomer).
Technologies
AWS S3, AWS DMS, AWS Redshift, AWS Athena, AWS Fargate, Airflow, Hudi, Spark Streaming, Flink, Kafka, Kinesis, RabbitMQ, Python, Java, Scala.
Responsibilities
Lead design and implementation of data pipelines; manage data architecture decisions; integrate 3rd party data sources; ensure data quality and storage optimization; collaborate with Data Scientists for model training.
Seniority
Senior, hands-on IC