SM/M, Senior Data Engineer
Core
Designing, building, and maintaining scalable data pipelines and infrastructure for IMDA's Data Platforms to enable data-driven decisions and AI-ready capabilities.
Role type
Senior Data Engineer with applied data science focus
Builds
Scalable batch and real-time data pipelines, data warehouse/lakehouse structures, and AI-ready datasets
Domain
Government data platforms, data engineering, applied data science
Deliverable
production ML models | infrastructure
Required skills
Python, SQL, Apache Spark, Airflow, data modeling, data quality, anomaly detection, MLOps, data governance, cloud platforms
Preferred skills
GCC environment experience, robotics/IoT sensor data, real-time streaming data, data cataloguing tools
Technologies
Apache Spark, Airflow, Python, SQL
Responsibilities
Design and maintain data pipelines for batch and real-time processing; Contribute to data modeling and architecture decisions; Apply data science knowledge for dataset preparation; Collaborate with stakeholders to translate requirements; Ensure compliance with regulatory requirements; Champion documentation and platform operability; Provide technical guidance and mentoring
Seniority
Senior, hands-on IC with mentoring responsibilities
