ETL Developer
Core
Designing, developing, and optimizing data pipelines to support a geospatial visualization platform and ensure reliable data flow into a PostgreSQL database.
Role type
Senior ETL Developer (Geospatial Data)
Builds
Data pipelines and ETL workflows for a geospatial visualization platform
Domain
Geospatial intelligence, Data Engineering
Deliverable
production ML models | product features
Required skills
ETL process design, SQL for large datasets, API integration, Python for data manipulation, PostgreSQL database design, Apache NiFi workflows, Git version control, AWS architecture, containerized development
Preferred skills
Apache Airflow, Java for custom NiFi processors, geospatial data tools, basic data science concepts
Technologies
Apache NiFi, PostgreSQL, AWS, Python, Git, Apache Airflow
Responsibilities
Design and manage ETL workflows using Apache NiFi; Monitor and troubleshoot real-time issues in NiFi and AWS/PostgreSQL infrastructure; Develop new NiFi flows adhering to best practices; Interface with analysts to comprehend data flow needs and refine requirements; Programmatically manipulate and transform data from multiple sources; Query and analyze large datasets using SQL; Retrieve and process data from external systems via APIs; Support data quality, integrity, and performance optimization efforts.
Seniority
Senior, hands-on IC