Lead Software Engineer
Core
Lead software development and modernization of large-scale batch data flows, processing, and indexing systems for global library datasets.
Role type
Senior IC Lead Software Engineer (Data Engineering)
Builds
Reliable, modernized batch processing pipelines, data indexes, and distributed processing solutions for global library networks.
Domain
Library technology, data engineering, batch processing
Deliverable
production ML models | product features | infrastructure
Required skills
Java, Bash scripting, Apache Spark, batch processing architecture, distributed systems design, technical leadership, system troubleshooting, data governance
Preferred skills
AI-assisted development tools, agile methodologies, cross-team collaboration
Technologies
Java, Spring, Apache Spark, Bash, enterprise scheduling systems
Responsibilities
Own and evolve large-scale data flows end-to-end; drive modernization of data-flow architecture; lead solution design for batch and distributed processing; develop and tune Apache Spark workloads; build and maintain scripting and orchestration at scale; establish technical standards for batch engineering; collaborate to align roadmaps with product needs; troubleshoot complex production issues in data flows.
Seniority
Senior, hands-on IC with mentorship responsibilities
