CareerPlan

Software Engineer, Data Infrastructure

USAFull-time2026-11-17 → 2026-11-18
On-site

Core

Design, build, and operate scalable distributed compute, data orchestration, and multimodal storage infrastructure for LLM research.

Role type

Senior IC data infrastructure engineer

Builds

High-throughput data ingestion, processing, transformation, training-data catalogs, deduplication, quality checks, and search systems for LLM research

Domain

Artificial Intelligence / Data Engineering

Deliverable

infrastructure

Required skills

Python, Rust, Apache Spark, Ray, cloud infrastructure, data lake architectures, batch pipelines, streaming pipelines

Preferred skills

Kafka, dbt, Terraform, Airflow, web crawling, data mining, search systems, Parquet, Delta Lake

Technologies

Apache Spark, Ray, Kafka, dbt, Terraform, Airflow, Parquet, Delta Lake

Responsibilities

Design and build distributed compute and data orchestration systems; Develop high-throughput data ingestion and processing pipelines; Implement monitoring and alerting for platform reliability; Collaborate with research teams to improve data quality and accelerate training cycles

Seniority

Senior, hands-on IC