Data Engineer - Search Department (SED)
Core
Design, develop, and maintain scalable data pipelines and systems to support search functionality, ensuring data is accessible, reliable, and optimized for improving search quality and relevance.
Role type
Data Engineer (Search Platform)
Builds
Scalable data pipelines, ETL processes, and data storage solutions for the Search Department
Domain
Internet services, E-commerce, Search technology
Deliverable
production ML models | product features | infrastructure
Required skills
SQL, Python, Java, Scala, Apache Spark, Apache Kafka, Apache Flink, Hadoop, ETL tools (Apache NiFi, Talend, Airflow), Data warehousing (Snowflake, Redshift, BigQuery), Data modeling, Data architecture, Data governance
Preferred skills
Containerization and orchestration (Docker, Kubernetes), Machine learning workflows, Semantic search, Image search, Data visualization (Tableau, Power BI)
Technologies
Apache Spark, Apache Kafka, Apache Flink, Hadoop, AWS, GCP, Azure, Snowflake, Redshift, BigQuery, Docker, Kubernetes, Tableau, Power BI
Responsibilities
Design and maintain scalable data pipelines; Implement and manage ETL processes; Optimize and maintain data storage solutions; Ensure data quality, integrity, and security; Monitor and troubleshoot data pipeline performance; Document data processes and workflows
Seniority
Mid-level to Senior, hands-on IC
