Senior Data Engineer
Core
Design, implement, and maintain ELT/ETL pipelines and data architecture in Azure Synapse and Azure Machine Learning to support SBA OIG investigations and audits.
Role type
Senior IC data engineer (cloud data platform)
Builds
Production data pipelines, data lakes, and self-service query capabilities for analysts
Domain
Public sector (SBA OIG) / Cloud data engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
ELT/ETL pipeline design, Azure Synapse, Azure Machine Learning, Python (Pandas), SQL/T-SQL, data normalization, source control, CI/CD workflows, Parquet optimization
Preferred skills
PySpark, Polars, Terraform/Bicep, entity resolution, AI coding assistants, LLM integration patterns
Technologies
Azure Synapse, Azure Machine Learning, Azure Data Lake Storage, Python, SQL, T-SQL, Parquet, Pandas, PySpark, Polars, Terraform, Bicep
Responsibilities
Design and maintain data architecture and pipelines; migrate source data to Azure Data Lake; normalize entity attributes; establish quality controls and error handling; optimize ingestion and storage; develop self-service data access for analysts; author SOPs and documentation; evaluate AI tooling for automation
Seniority
Senior, hands-on IC