AI Data Engineer
Core
Design, build, and maintain robust data pipelines and infrastructure to power SchonAI, an internal AI platform serving investment professionals.
Role type
Senior AI Data Engineer
Builds
Scalable data pipelines, ETL/ELT processes, and AI/ML data infrastructure (vector databases, semantic search systems)
Domain
Financial services / AI Data Engineering
Deliverable
production ML models
Required skills
Python, SQL, Apache Airflow/Prefect/Dagster, Apache Spark/Flink, AWS (S3, Kubernetes), PostgreSQL/MySQL, MongoDB/DynamoDB/Elasticsearch, vector databases (Pinecone/Weaviate/Qdrant)
Preferred skills
LLM/RAG pipeline experience, financial data sources, Kafka/Kinesis/Pub/Sub, OLAP DBs (BigQ/SingleStore/RedShift/ClickHouse), Docker/Kubernetes, MLOps, data quality frameworks (Great Expectations/Deequ)
Technologies
Prefect, Apache Airflow, Apache Spark, Apache Flink, AWS S3, AWS Kubernetes, PostgreSQL, MySQL, MongoDB, DynamoDB, Elasticsearch, Pinecone, Weaviate, Qdrant, Kafka, Kinesis, Docker, RedShift, ClickHouse
Responsibilities
Design and build scalable data pipelines for structured and unstructured data; Develop ETL/ELT processes for diverse data sources; Implement real-time and batch data processing workflows; Build and maintain data infrastructure optimized for AI/ML workloads including vector databases; Partner with AI engineers and data scientists to understand data requirements; Implement data access controls, encryption, and compliance measures
Seniority
Senior, hands-on IC

