Senior Data Engineer / Data Curator
Core
Design and maintain scalable data pipelines to curate high-quality datasets for AI model training and deployment, ensuring data integrity, bias mitigation, and compliance.
Role type
Senior IC data engineer (AI data curation)
Builds
Production-ready training datasets and data pipelines for AI/ML models
Domain
Semiconductor manufacturing / AI Data Engineering
Deliverable
production ML models
Required skills
Python (Pandas, NumPy), SQL, cloud data storage (AWS S3, GCS), distributed systems, data annotation tools, NLP data formats (JSONL, embeddings, tokenization), ETL tools (Apache Kafka, Airflow), AI ethics and compliance (GDPR, CCPA)
Preferred skills
Vector databases and indexing for LLMs (FAISS, Pinecone)
Responsibilities
Design and implement data pipelines for processing and curating large datasets; Automate data cleaning and labeling processes; Collaborate with model teams to align data with performance goals; Assess and mitigate bias in datasets; Manage data storage and retrieval strategies; Conduct audits for data integrity and privacy compliance
Seniority
Senior, hands-on IC