COSMOS - Data Engineer IV
Core
Lead a team of data engineers to collect, manage, and convert raw data (including social media) into usable information for analytics, decision-making, and cutting-edge social computing research.
Role type
Senior IC Data Engineer (Team Lead)
Builds
Data pipelines, microservices, visualization dashboards, and ML model infrastructure for research projects.
Domain
Academic Research / Social Computing / Data Engineering
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
Team leadership, ETL pipeline design, NoSQL storage, Microservices architecture, Data modeling, CI/CD, Front-end development, ML model training/deployment
Preferred skills
Social media data collection, Real-time streaming, Legacy database migration
Technologies
Kafka, MongoDB, Kubernetes, Python, Golang, FlaskAPI, GraphQL, Docker, Elasticsearch, Grafana, Prometheus, Kibana, Jenkins, GitLab CI, Spring Boot, React, HTML/CSS, JavaScript, Node.js, SQL
Responsibilities
Lead a team of 5+ data engineers; Design and manage data ETL pipelines processing 10M+ data points daily; Design and deploy scalable microservices; Migrate large datasets to MongoDB; Create CI/CD pipelines; Integrate monitoring and visualization tools; Train and deploy ML models; Develop analytical tools and front-end interfaces.
Seniority
Senior, hands-on IC with leadership responsibilities