移动抓取与收录研发工程师（J92916）
Core
Building a distributed crawler system for massive data collection and overcoming technical challenges in high-performance, high-concurrency, and high-availability scenarios.
Role type
Senior IC crawler engineer (distributed systems)
Builds
Distributed data crawling infrastructure
Domain
Internet technology / Web scraping / Reverse engineering
Required skills
C/C++/Python, HTTP/HTTPS protocols, Web scraping frameworks (Scrapy, BeautifulSoup, Selenium), browser rendering principles, reverse engineering (APP/Web), anti-crawling strategy bypass, risk control identification
Preferred skills
Problem-solving for challenging technical scenarios
Technologies
Scrapy, BeautifulSoup, Selenium, C++, Python, HTTP, HTTPS
Responsibilities
Develop and maintain distributed crawler systems; solve technical challenges in high-concurrency environments; write high-quality code and documentation; overcome specific crawling obstacles
Seniority
Senior, hands-on IC