Senior Data Acquisition Engineer at JoomPulse
Core
Building and maintaining distributed data acquisition pipelines to scrape product data from e-commerce marketplaces like Mercado Livre and Shopee for an AI analytics tool.
Role type
Senior IC data acquisition engineer (web scraping/crawling)
Builds
Distributed crawlers and data pipelines for product insights
Domain
E-commerce data acquisition and anti-bot evasion
Deliverable
production ML models
Required skills
Python (Scrapy, Playwright) or Node.js (Puppeteer, Crawlee), behavioral simulation, proxy infrastructure management, API reverse engineering, distributed queue scaling, LLM usage
Preferred skills
Experience with Mercado Livre/Shopee/TikTok Shop, anti-detect browsers, project end-to-end ownership
Technologies
Scrapy, Playwright, Puppeteer, Crawlee, Redis, Kafka, SQS, mitmproxy, Charles, Burp, Multilogin, GoLogin
Responsibilities
Develop scraping for Mercado Livre and extend to new marketplaces, build and maintain distributed crawlers, monitor marketplaces for anti-bot changes, reverse engineer private APIs, keep crawlers scalable and observable, investigate pipeline failures, use LLMs to solve novel problems, spot useful data for the product
Seniority
Senior, hands-on IC
