Senior Software Engineer (Python + Distributed systems)
Core
Design and optimize large-scale event-driven, distributed data pipelines on AWS to extract, enrich, and process metadata from hundreds of millions of documents and billions of images for Scribd's content discovery systems.
Role type
Senior Software Engineer (Python + Distributed Systems)
Builds
Scalable backend services and metadata enrichment pipelines for Scribd, Everand, and Slideshare products.
Domain
Internet / Document Processing / Distributed Systems
Deliverable
production ML models | product features | infrastructure
Required skills
Python (5+ years), AWS (ECS, Lambda, SQS, ElastiCache), Terraform, System Architecture, Event-driven design, Scalability optimization, API development, Observability, Technical leadership
Preferred skills
Scala, Spark, Databricks, ML/LLM integration
Technologies
Python, AWS (Lambda, ECS, SQS, ElastiCache, CloudWatch), Airflow, Spark, Databricks, Terraform, Datadog
Responsibilities
Lead design and scaling of event-driven distributed systems for metadata extraction; Partner with Data Science and ML teams to architect robust systems; Build and maintain high-throughput APIs and backend services; Optimize existing systems for performance and reliability; Ensure system health via monitoring and automated testing; Mentor engineers and drive secure coding practices.
Seniority
Senior, hands-on IC with leadership responsibilities
