Senior HPC and Platform Engineer
Core
Design and build a distributed execution framework that runs scientific workloads consistently across local workstations, on-premise HPC, managed services (Databricks, Spark), and cloud batch environments (AWS, Azure, GCP) behind a single interface.
Role type
Senior HPC and Platform Engineer
Builds
Distributed execution framework, build and delivery pipelines, observability systems
Domain
Geoscience software, High-Performance Computing, Cloud Infrastructure
Deliverable
production ML models
Required skills
Distributed compute frameworks (Ray, Dask, Spark, MPI), Python, C++, Infrastructure as Code (Terraform), Kubernetes, Cloud providers (AWS, Azure, GCP), Blob/Object storage performance, Cloud security (IAM, secrets, encryption), CI/CD pipelines
Preferred skills
HPC schedulers (Slurm, PBS), Managed cloud batch/Spark services, GPU computing (CUDA), Cloud-native chunked array storage, OSDU integration, Security certifications, AI coding tools
Technologies
Ray, Dask, Spark, MPI, Terraform, Kubernetes, AWS, Azure, GCP, Databricks, Slurm, PBS, CUDA
Responsibilities
Design task sequencing, elastic scaling, checkpoint/restart, and error reporting for distributed execution; Build job admission, entitlement, and metering paths; Implement security surface including identity management and network isolation; Profile and optimize compute-bound and I/O-bound scientific workloads; Own end-to-end CI/CD and release engineering; Establish performance baselines and observability across backends
Seniority
Senior, hands-on IC