Data Engineer
Core
Build, maintain, and improve batch and streaming data pipelines for enterprise data warehouses and lakehouse platforms, supporting AI knowledge bases with RAG pipelines.
Role type
Senior Data Engineer (AI/ML Infrastructure)
Builds
Enterprise data warehouses, lakehouse platforms, real-time streaming pipelines, and AI knowledge bases.
Domain
Financial Services / Data Engineering & AI Infrastructure
Deliverable
production ML models
Required skills
SQL, Python, Spark, ETL, Airflow, Docker, Kubernetes, Terraform, REST APIs, Vector Databases, RAG pipelines
Preferred skills
Flink, CDC tools (Debezium, Confluent), Redis, LLM/AI-agent use cases, real-time system design, Banking experience
Technologies
Cloudera, GCP, AWS, Airflow, Dagster, Flask, Redis, Confluent, Debezium, Fivetran
Responsibilities
Develop reusable data models and ingestion pipelines from APIs and GA4; Create and support batch processing, real-time streaming, and CDC workflows; Deploy and support data workloads on cloud infrastructure; Build REST APIs and backend services; Support vector database and RAG pipelines for AI knowledge bases.
Seniority
Senior, hands-on IC

