Data Scientist
Core
Build and maintain bulk data pipelines, process OS and data logs into metrics dashboards, and support new system capabilities in customer cloud environments.
Role type
Senior Data Engineer / Data Scientist (ETL & Analytics)
Builds
Enriched, query-friendly structured data in indexed files; analytic reports and metrics dashboards.
Domain
Cloud computing, Big Data, Log Analysis
Deliverable
production ML models | dashboards & analysis
Required skills
Python, Spark, Pyspark, SQL, MySQL, PostgreSQL, ETL, Data Modeling, Regular Expressions, XML/JSON processing, CloudWatch, Kibana, ElasticSearch, Logstash, DevOps tools, Pandas
Preferred skills
Databricks, Elastic Map Reduce, Spark streaming/batch tuning, AWS services (Lambda, SNS/SQS, EC2, Dynamo, RDS), Notebook-based pipelines
Technologies
Spark, Pyspark, Python, SQL, MySQL, PostgresSQL, CloudWatch, Kibana, ElasticSearch, Logstash, Databricks, AWS (Lambda, SNS, SQS, EC2, Dynamo, RDS), ELK stack
Responsibilities
Support requirements for new system capabilities; extend and maintain bulk data pipelines; process and convert OS and data logs into reports and metrics dashboards; develop ETL design documentation; interface with customers and integration partners; support Agile development.
Seniority
Senior, hands-on IC
