Senior Data Engineer
Core
Design, develop, and maintain scalable data pipelines and storage solutions for sensor data to support AI model training, inference, and analytics for infrastructure monitoring.
Role type
Senior Data Engineer
Builds
Scalable data pipelines, data access services, and optimized storage solutions for structured, semi-structured, and high-dimensional sensor data.
Domain
Infrastructure monitoring, optical fibers, AI/ML data infrastructure
Deliverable
production ML models
Required skills
Python, Apache Airflow, PostgreSQL, Kafka, AWS Kinesis, FastAPI, Docker, CI/CD
Preferred skills
Kubernetes, distributed computing platforms, ML data pipelines, time-series data, Grafana, Prometheus
Technologies
Apache Airflow, PostgreSQL, MySQL, MongoDB, Elasticsearch, ClickHouse, AWS Kinesis, Amazon SQS, RabbitMQ, Kafka, FastAPI, AWS S3, EC2, Lambda, CloudWatch, IAM, Git, Docker, Grafana, Prometheus, ELK, Kibana, OpenSearch
