Data Engineering
Core
Architect high-performance data infrastructure and real-time pipelines enabling agentic AI to reason over complex enterprise environments.
Role type
AI Data Engineer
Builds
Real-time data pipelines, vector databases, semantic layers, and knowledge graphs for autonomous systems.
Domain
Enterprise IT, AI/ML infrastructure, Data Engineering
Deliverable
production ML models
Required skills
Data mining, ETL processes, data pipeline development, relational and NoSQL databases, vector database management, semantic layer design, automated data guardrails, CI/CD pipeline management
Preferred skills
Cloud modernization, Data Modelling, professional data engineering certifications, cloud platform certifications, social coding and IDE usage
Technologies
Glue, Databricks, Synapse, Dataproc, PostgreSQL, DB2, MongoDB, Pinecone, Milvus, Weaviate, GitHub, Visual Studio
Responsibilities
Design and scale RAG pipelines for unstructured IT logs and documentation; ensure health and performance of vector databases; build knowledge graphs and semantic layers; automate data quality checks for PII and bias; resolve data quality issues at the source; build and maintain CI/CD pipelines for data infrastructure
Seniority
Mid-to-Senior, hands-on IC