CareerPlan

AI Data Engineer

New York CityFull-time$225k–$275k2026-11-15
On-siteNo visa sponsorship

Core

Design, build, and maintain robust data pipelines and infrastructure to power SchonAI, an internal AI platform serving investment professionals.

Role type

Senior AI Data Engineer

Builds

Scalable data pipelines, ETL/ELT processes, and AI/ML data infrastructure (vector databases, semantic search systems)

Domain

Financial services / AI Data Engineering

Deliverable

production ML models

Required skills

Python, SQL, Apache Airflow/Prefect/Dagster, Apache Spark/Flink, AWS (S3, Kubernetes), PostgreSQL/MySQL, MongoDB/DynamoDB/Elasticsearch, vector databases (Pinecone/Weaviate/Qdrant)

Preferred skills

LLM/RAG pipeline experience, financial data sources, Kafka/Kinesis/Pub/Sub, OLAP DBs (BigQ/SingleStore/RedShift/ClickHouse), Docker/Kubernetes, MLOps, data quality frameworks (Great Expectations/Deequ)

Technologies

Prefect, Apache Airflow, Apache Spark, Apache Flink, AWS S3, AWS Kubernetes, PostgreSQL, MySQL, MongoDB, DynamoDB, Elasticsearch, Pinecone, Weaviate, Qdrant, Kafka, Kinesis, Docker, RedShift, ClickHouse

Responsibilities

Design and build scalable data pipelines for structured and unstructured data; Develop ETL/ELT processes for diverse data sources; Implement real-time and batch data processing workflows; Build and maintain data infrastructure optimized for AI/ML workloads including vector databases; Partner with AI engineers and data scientists to understand data requirements; Implement data access controls, encryption, and compliance measures

Seniority

Senior, hands-on IC