Principal Software Architect - Data Platform
Core
Lead system design for an internet-scale data platform that ingests, processes, and serves analytics for 12 million companies' cybersecurity ratings.
Role type
Principal Software Architect (Data Platform)
Builds
The core data platform serving as the product for cybersecurity ratings, including ingestion, processing, storage, and analytical serving layers.
Domain
Cybersecurity / Data Engineering / Distributed Systems
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
Large-scale data platform architecture, Stream and batch processing (Kafka, Flink, Spark), Python, PySpark, Java, Scala, Lakehouse storage design (Parquet, Iceberg), OLAP/Analytical serving layer design (ClickHouse, Druid, Pinot, BigQuery, Snowflake), Distributed systems fundamentals, Data quality engineering, Technical writing and mentorship
Preferred skills
Data layer for ML/LLM systems, Internet-scale telemetry/observability data, Platform cost optimization
Technologies
Kafka, Flink, Spark, Python, PySpark, Java, Scala, Parquet, Iceberg, ClickHouse, Druid, Pinot, BigQuery, Snowflake, Node.js, TypeScript, React, PostgreSQL, AWS, Kubernetes, Terraform, Helm, ArgoCD, C++