Data Engineer
Core
Design, develop, and maintain robust batch and streaming pipelines to ingest data from cloud and on-premise sources into a lakehouse platform, ensuring high data integrity for business intelligence and machine learning.
Role type
Senior Data Engineer
Builds
Scalable data warehouse solutions and data products for Finance, Customer Service, Sales, and Data Science teams.
Domain
Travel technology and fintech
Deliverable
production ML models
Required skills
Data warehousing concepts (Star/Snowflake schemas, SCDs), Batch and streaming pipeline development, Distributed query engines (Spark, Trino/Dremio), Orchestration (Airflow), SQL, Python, Scala, Bash, Data lakehouse architectures (Iceberg, Delta Lake), Distributed messaging (Kafka), CDC tools (Debezium, Flink), Linux, Docker, Kubernetes, CI/CD, Monitoring (Grafana, Kibana)
Preferred skills
Secrets management (Vault)
Technologies
Spark, dbt, Dremio/Trino, Databricks, Airflow, Kafka, Debezium, Flink, S3/Ceph, Apache Iceberg, Delta Lake, Parquet, Avro, JSON, CSV, Jenkins, Artifactory, Grafana, Kibana, Vault
Responsibilities
Design and maintain batch and streaming pipelines; Ingest data from diverse source systems into the data lakehouse; Evaluate and configure industry-leading frameworks for the data platform; Define and enforce data quality standards with monitoring and alerting; Collaborate with ML Engineers, backend developers, and stakeholders to optimize pipelines; Apply performance optimization techniques (indexing, partitioning, caching, data modeling).
