Senior Data Engineer, Customer-Facing Data Products
Core
Building real-time data pipelines and APIs to process events and serve aggregated data for customer-facing use cases like behavioral analysis and targeting.
Role type
Senior IC data engineer (real-time/streaming)
Builds
Reusable real-time data pipelines, event-driven architectures, and high-availability data serving APIs for customer-facing products.
Domain
Media & Publishing / Real-time Data Engineering
Deliverable
production ML models | product features | infrastructure
Required skills
Event-driven architectures, stream-processing frameworks, cloud native architectures (AWS), API design (REST, GraphQL, gRPC), Python, version control, CI/CD tools
Preferred skills
Apache Kafka, Apache Flink, Spark Streaming, Python frameworks (FastAPI, Flask), SQL, data lakehouse and medallion architectures
Technologies
AWS, Apache Kafka, Apache Flink, Spark Streaming, FastAPI, Flask, Github Actions, Drone
Responsibilities
Develop real-time data pipelines using event-driven architectures and streaming technologies; Ingest and organize structured and unstructured data; Engineer and scale high availability data serving capabilities; Implement mechanisms to ensure data quality, observability and governance best practices; Collaborate with software engineers and infrastructure teams to improve pipeline performance; Grow the skills of colleagues by providing clear technical feedback through pairing, design, and code review
Seniority
Senior, hands-on IC