Data Engineer (Python)
Core
Designing and managing large enterprise data and analytics platforms, including scalable data lakes, ingestion pipelines, and ML/NLP analytics.
Role type
Senior Data Engineer
Builds
Scalable Smart Data Lakes, Data Ingestion Platforms, Machine Learning and NLP based Analytics Platforms, Hyper-Scale Processing Clusters, Data Mining and Search Engines
Domain
Big Data, Data Engineering, Machine Learning, NLP
Deliverable
production ML models | infrastructure
Required skills
Python, PySpark, Pandas, SQL, NoSQL, Stream Processing (Kafka, Spark-Streaming), Event-driven architectures, RESTful APIs
Preferred skills
Pytest, Docker, Kubernetes
Technologies
PySpark, Pandas, Kafka, Spark-Streaming, Postgres, MongoDB, Elasticsearch
Responsibilities
Create and manage end-to-end data solutions and optimal data processing pipelines for large volume, varied data sets; Develop efficient pre-processing and data manipulation tasks; Implement and manage scalable data lakes and analytics platforms; Work with data science and infrastructure teams to implement practical machine learning solutions in production.
Seniority
Mid-Senior, hands-on IC
