Engineer II (R-20095)
Core
Build and maintain a self-developed data collection system for high-value corporate business registration data, focusing on incremental updates and validation for up to 10,000 entities daily.
Role type
Senior IC data engineering engineer (web scraping & API reverse engineering)
Builds
Stable, monitored data collection pipelines and structured data ingestion for corporate master files
Domain
Data engineering / Web scraping / Corporate information services
Required skills
Python, Playwright, browser automation, API reverse engineering, JavaScript deobfuscation, proxy management, task scheduling, monitoring & alerting
Preferred skills
WebSocket/GraphQL handling, RPA integration, cost optimization under resource constraints
Technologies
Python, Playwright, Chrome DevTools, Requests, httpx, Webpack, Vite, ODS
Responsibilities
Design and implement browser automation services for data collection; analyze and reverse engineer frontend interfaces and API signatures; handle anti-scraping mechanisms like CAPTCHAs and IP bans; ensure 7x24 stability and data quality; collaborate on data governance and multi-source validation; optimize collection strategies and resource usage.
Seniority
Senior, hands-on IC