AI Data Engineer II
Core
Building and supporting scalable data pipelines and cloud-based data solutions, with exposure to AI and Generative AI use cases.
Role type
mid-level data engineer (cloud & AI data)
Builds
ETL/ELT pipelines, data transformation solutions, enterprise datasets, and data models for reporting and downstream applications
Domain
Cloud data engineering, AI data preparation, enterprise data platforms
Required skills
Python, SQL, PySpark, Databricks, ETL/ELT pipeline development, cloud data platforms (Azure), data validation and quality checks, Git/CI-CD
Preferred skills
Generative AI, LLMs, embeddings, semantic search, vector databases, RAG, AI-assisted automation
Technologies
GCP, Azure Data Lake, Databricks, PySpark, SQL, Python, Git
Responsibilities
Develop and maintain ETL/ELT data pipelines for batch processing; Build data transformation solutions; Work with Databricks to ingest, transform, and curate enterprise datasets; Integrate data from relational databases, APIs, files, and cloud storage; Support migration of datasets to modern cloud platforms; Perform data validation and quality checks; Develop and maintain datasets and data models; Troubleshoot pipeline failures and performance issues; Participate in code reviews and CI/CD practices; Collaborate with data scientists and business teams; Support AI-related data requirements including preparing datasets for LLMs and RAG.
Seniority
Mid-level, hands-on IC
