CareerPlan

Lead Data Engineer

BCC-BethesdaRemoteFull-time2026-10-20 → 2026-11-19

Core

Design and implement a modern lakehouse data platform for a federal permitting modernization initiative, leading engineering patterns for ingestion, harmonization, and serving data across multiple source systems.

Role type

Lead Data Engineer (hands-on technical leadership)

Builds

Core analytical data platform (DuckDB/Apache Iceberg on AWS), certified data products, and business metrics

Domain

Federal Government / Data Engineering / Cloud Infrastructure

Required skills

DuckDB, Apache Iceberg, AWS (S3, Glue), Python, Advanced SQL, Data Modeling (Bronze/Silver/Gold), Metadata-driven engineering, AI/Agentic workflows (LangChain/LangGraph), Technical Leadership

Preferred skills

Kafka/MSK, GeoParquet/PostGIS, OpenSearch, Enterprise SaaS integration, U.S. Federal Government experience

Technologies

DuckDB, Apache Iceberg, Amazon S3, AWS Glue, Python, LangChain, LangGraph

Responsibilities

Design and implement the core DuckDB / Apache Iceberg data platform on AWS; Define physical data organization, schema, partitioning, file layout, snapshot, compaction, and optimization strategies for Iceberg tables; Integrate DuckDB with Amazon S3, Apache Iceberg, and AWS Glue Catalog; Establish bronze, silver, and gold data-layer conventions and reusable engineering patterns; Lead and mentor Data Engineers, providing technical direction, code reviews, and engineering guidance; Build and maintain analytical datasets supporting application services, reporting, analytics, and data science; Develop reusable patterns for incremental loads, CDC, schema evolution, late-arriving data, backfills, and historical reconstruction; Work with the Data Architect to implement the common canonical data model and source-to-canonical mappings; Lead migration and harmonization of datasets from multiple source systems; Implement reconciliation processes that demonstrate completeness and accuracy between source and target systems; Establish automated data-quality monitoring, operational observability, and testing; Implement agentic and AI-enabled data engineering patterns supporting activities such as source analysis, mapping, pipeline generation, troubleshooting, reconciliation, and quality assurance; Support integration of structured data with associated document and geospatial datasets; Lead development of certified data products, analytical datasets, and business metrics; Optimize DuckDB queries and Iceberg datasets for performance and efficiency; Develop engineering documentation, operational procedures, and technical handoff materials.

Seniority

Lead, hands-on technical leadership