Senior Data Engineer
Core
Build and scale high-throughput streaming and batch data pipelines to power real-time security insights and actionable intelligence for engineering and ML teams.
Role type
Senior hands-on IC data engineer
Builds
High-throughput data pipelines, robust APIs, and optimized data storage solutions
Domain
Cybersecurity / Financial Services / Insurance
Deliverable
production ML models | infrastructure
Required skills
Apache Beam, Dataflow, BigQuery, Bigtable, Python, FastAPI, asynchronous programming, automated testing (pytest, Ruff), pipeline engineering, data schema design
Preferred skills
PySpark, linting tools
Technologies
Apache Beam, Dataflow, BigQuery, Bigtable, Python, FastAPI, Poetry, Ruff, pytest
Responsibilities
Build and scale high-throughput streaming pipelines; Optimize data flows into BigQuery and Bigtable; Develop and maintain Python-based data APIs; Investigate complex data anomalies to solve root causes; Partner with ML/Product teams to build data schemas; Maintain high-quality production Python code
Seniority
Senior, hands-on IC
