Senior Backend Engineer (Python) - Content Understanding
Core
Design and optimize large-scale event-driven, distributed data and service pipelines to extract, enrich, and process metadata from hundreds of millions of documents and billions of images for Scribd's content discovery systems.
Role type
Senior Backend Engineer (Python) specializing in distributed systems and metadata processing
Builds
Scalable metadata extraction and enrichment pipelines for ebooks, audiobooks, and user-generated content
Domain
Digital publishing, machine learning infrastructure, distributed systems
Deliverable
production ML models | infrastructure
Required skills
Python, event-driven architecture, distributed systems design, AWS services (ECS, Lambda, SQS, ElastiCache), Terraform, system performance optimization, technical leadership
Preferred skills
Scala, Spark, Databricks, workflow orchestration (Airflow), integrating ML/LLM models into production
Technologies
Python, AWS (Lambda, ECS, SQS, ElastiCache, CloudWatch), Airflow, Spark, Databricks, Terraform, Datadog
Responsibilities
Lead design and scaling of event-driven distributed systems for metadata processing; Partner with Data Science and ML Engineering teams to architect robust systems; Build and maintain scalable APIs and backend services for high-throughput content processing; Optimize and refactor existing backend systems for scalability and reliability; Ensure system health and data integrity through monitoring and automated testing
Seniority
Senior, hands-on IC with leadership responsibilities
