Data Engineer
Core
Building and operating production AI data systems, including ingestion pipelines, storage design, and retrieval infrastructure for training models and RAG queries.
Role type
Senior IC Data Engineer
Builds
Ingestion/transformation pipelines, data warehouses/lakehouses, and retrieval layers for AI systems
Domain
AI/ML Data Infrastructure
Deliverable
production ML models
Required skills
Python, SQL, Spark, Kafka, dbt, Airflow, Snowflake, BigQuery, AWS, Azure, Docker, CI/CD, Terraform, Vector databases (pgvector, Pinecone)
Preferred skills
Streaming workloads, Managed cloud services, Jupyter notebooks, SOC 2/HIPAA compliance, Open-source contributions
Technologies
Spark, Kafka, dbt, Airflow, Snowflake, BigQuery, Redshift, Databricks, pgvector, Pinecone, Qdrant, Azure AI Search, AWS, Azure
Responsibilities
Design and build batch/streaming ingestion pipelines with idempotency and schema evolution; Architect warehouse and lakehouse storage solutions; Implement chunking, embedding, and indexing pipelines for RAG systems; Establish data quality tests, lineage, and alerting; Manage PII classification, masking, and access controls; Operate containerized deployments with CI/CD and cost monitoring
Seniority
Senior, hands-on IC
