COSMOS - Data Engineer III
Core
Lead a team of data engineers to collect, manage, and convert raw data (including social media) into usable information for analytics, decision-making, and cutting-edge social computing research.
Role type
Senior IC data engineer (team lead)
Builds
Data pipelines, microservices, ML models, and visualization dashboards for research projects
Domain
Academic research / Social computing
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Team leadership, data pipeline design, ETL processes, NoSQL storage, microservices architecture, ML model training/deployment, CI/CD, front-end development
Preferred skills
Social media data collection, real-time streaming, system fault tolerance
Technologies
Kafka, MongoDB, Kubernetes, Python, Golang, FlaskAPI, GraphQL, Docker, Elasticsearch, Grafana, Prometheus, Kibana, Jenkins, GitLab CI, Node.js
Responsibilities
Lead a team of data engineers; Design and manage data ETL pipelines processing 10M+ data points daily; Design and deploy scalable microservices; Train and deploy ML models; Create visualization dashboards; Collaborate with researchers on social computing projects
Seniority
Senior, hands-on IC with leadership responsibilities
