Data Warehouse / BI Architect
Core
Designing, implementing, and optimizing data ingress/egress pathways and dataflow design for analytics workloads, including preprocessing, validation, and anomaly detection for structured, semi-structured, and unstructured datasets.
Role type
Senior IC Data Warehouse / BI Architect
Builds
Scalable and reliable data processing pipelines for downstream analytics and machine learning
Domain
Data Engineering / Big Data
Deliverable
production ML models
Required skills
Python, Java, Apache Spark, Apache Airflow, SQL (MySQL, MariaDB, PostgreSQL), Linux CLI, Bash scripting, Git, pandas, NumPy, data wrangling, data validation, anomaly detection
Preferred skills
HPC Job Scheduling (Slurm), Atlassian Tool Suite (JIRA, Confluence)
Technologies
Apache Spark, Apache Airflow, MySQL, MariaDB, PostgreSQL, Linux, Bash, Jupyter Notebook, Parquet, JSON, CSV, XML, Git, Slurm, JIRA, Confluence
Responsibilities
Design and implement data ingress/egress pathways; Prepare datasets for downstream analytics and ML; Optimize data processing for efficiency and scalability; Orchestrate complex workflows using Airflow; Perform data validation and anomaly detection.
Seniority
Senior, hands-on IC