Data / Integration Engineer
Core
Design, develop, and maintain scalable document ingestion and processing pipelines to support enterprise knowledge management and AI-powered applications.
Role type
Senior IC data integration engineer (AI/ML pipelines)
Builds
Document processing pipelines, vector database indexing strategies, MCP-compatible tool servers, and data analysis agents
Domain
Enterprise knowledge management, AI-driven data processing, cloud-native platforms
Deliverable
production ML models | product features
Required skills
Python, ETL/ELT workflow orchestration, document processing frameworks, vector databases, embedding models, REST API design, containerization (Docker), event-driven architectures
Preferred skills
RAG architectures, event-driven system design, AI agent ecosystems, MCP integrations
Technologies
Dagster, Airflow, Azure AI Search, Blob Storage, Event Hub, Document Intelligence, Docker
Responsibilities
Design and optimize multi-stage data workflows for diverse document formats; implement metadata extraction, content chunking, classification, and summarization; manage vector database indexing and search optimization; develop MCP-compatible tool servers; create and deploy data analysis and knowledge retrieval agents
Seniority
Senior, hands-on IC