Middle Data Engineer
Core
Own the analytical data infrastructure for JustDone, an AI assistant for academic audiences, ensuring stability and optimization while contributing to NLP tasks.
Role type
Middle Data Engineer
Builds
Production data pipelines, data warehouses, and vector databases supporting AI/NLP features.
Domain
EdTech / AI / Data Engineering
Deliverable
production ML models
Required skills
Python, SQL, Data Warehousing, Cloud Platforms, Orchestration, Data Quality, Vector Databases
Preferred skills
NLP infrastructure, Vector DBs (Milvus, Pinecone, Weaviate, Qdrant)
Technologies
BigQuery, Airflow, RabbitMQ, Kafka, Milvus, GCP
Responsibilities
Develop and maintain event pipelines from RabbitMQ to DWH; Integrate new data sources and develop ETL/ELT pipelines; Manage BigQuery and Airflow access and schemas; Model and optimize data architecture (Bronze/Silver/Gold); Ensure data quality and reliability with monitoring and anomaly alerts; Support NLP infrastructure by managing data loading pipelines and vector indexes.
Seniority
Middle, hands-on IC