数据研发工程师（J104987）
Core
Build and optimize large-scale offline and real-time data processing pipelines for Baidu Netdisk and overseas cloud storage, focusing on efficiency, stability, and intelligent governance using LLMs/Agents.
Role type
Senior IC Data Engineering (Big Data & AI)
Builds
Data processing infrastructure, intelligent SQL optimization tools, and automated data research assistants
Domain
Internet / Big Data / AI
Deliverable
production ML models | product features
Required skills
Big Data technologies (Hadoop, Hive, Spark, Flink, Kafka, OLAP), SQL performance tuning, Distributed computing, Java/Python/Shell programming, LLM/Agent application
Preferred skills
Slow query governance, Compute cost optimization, LLM for SQL optimization and task diagnosis
Technologies
Hadoop, Hive, Spark, Flink, Kafka, OLAP, LLM, Agent
Responsibilities
Design and optimize large-scale data processing chains including scheduling and resource governance; Upgrade data computing architecture and migrate legacy pipelines; Build intelligent governance capabilities for slow queries and resource waste using LLMs/Agents; Develop data R&D assistants and automation tools for task diagnosis and anomaly attribution; Platformize and automate frequent data R&D issues to reduce manual effort
Seniority
Senior, hands-on IC