Software Engineer, Data Platform
Core
Designing and implementing scalable data ingestion, processing pipelines, and resilient data storage solutions that power crawling, ranking, and retrieval systems for AI applications.
Role type
Senior IC software engineer (data platform & infrastructure)
Builds
Large-scale data infrastructure, ingestion connectors, and high-performance APIs
Domain
AI/ML infrastructure, cloud computing, data engineering
Deliverable
production ML models
Required skills
Python, SQL, distributed systems, NoSQL, cloud infrastructure (AWS/GCP/Azure), ETL pipeline orchestration (Airflow), containerization (Docker/Kubernetes), system monitoring and alerting
Preferred skills
Stream processing frameworks, telemetry tools, LLM application knowledge
Technologies
AWS, GCP, Azure, Airflow, Docker, Kubernetes, RESTful APIs
Responsibilities
Designing and implementing scalable data ingestion and processing pipelines; Develop high-performance RESTful APIs to facilitate seamless data access; Architect resilient data storage solutions using modern cloud technologies; Manage cloud infrastructure to support high-volume data processing; Implement automated monitoring, logging, and alerting systems; Evaluate and implement new technologies to improve efficiency
Seniority
Senior, hands-on IC