Software Engineer - Data Platform (Staff+)
Core
Design and implement foundational pillars of a data platform and integration pipeline to ingest, normalize, and transform enterprise data into a knowledge base for GenAI systems.
Role type
Staff-level Data Platform Engineer
Builds
Enterprise data ingestion pipelines, connectors, data lakes, and indexing strategies for semantic search and RAG.
Domain
Cybersecurity, Applied AI, Enterprise Data Infrastructure
Deliverable
production ML models | infrastructure
Required skills
Data pipeline architecture, ETL orchestration, distributed computing, database fundamentals, Kubernetes, Terraform, Databricks, Kafka, Flink
Preferred skills
Knowledge graph construction, semantic search optimization, build vs buy decisions for LLM stacks, scaling integration patterns
Technologies
Apache Kafka, Apache Flink, Kubernetes, Terraform, Docker, Databricks, Temporal, Airflow
Responsibilities
Build data connectors and run pipelines to extract insights from enterprise environments; Design performant data pipelines and indexing strategies for semantic search and retrieval augmented generation; Architect the modern data platform for Gen AI; Implement generalizable design patterns to scale from 1 to thousands of integrations; Make build vs buy decisions for LLM stack tools and services.
Seniority
Staff, hands-on IC with technical leadership
