CareerPlan

[Job-32448] Master Data Developer (Databricks), Brazil

BrazilFull-time2026-10-27 → 2026-11-19

Core

Design, develop, and maintain scalable data infrastructure and pipelines, specifically migrating data architectures from Databricks to AWS Glue and optimizing data quality and performance.

Role type

Senior Data Developer (Data Engineering)

Builds

Scalable data pipelines, ETL/ELT processes, data lakes, and warehouses

Domain

Data Engineering, Cloud Platforms (AWS), Big Data

Required skills

Databricks migration, Unity Catalog to Glue Catalog, Deltalake to Iceberg format, Spark SQL, PySpark, Python, AWS S3, AWS Glue, AWS Secrets Manager, AWS IAM, AWS Athena, AWS LakeFormation, AWS EMR, Auto Loader, DLT pipelines, Data quality validation, API development

Preferred skills

Databricks Certified Consultant, DBR to EMR on AWS migration experience

Technologies

Databricks, AWS, Spark, Python, Java, Scala, Iceberg, Delta Lake, S3, Glue, EMR, Athena, LakeFormation

Responsibilities

Build and optimize scalable data pipelines and ETL/ELT processes; Implement data architectures including data lakes and warehouses; Ensure data quality and governance through validation and monitoring; Develop integrations between data sources and analytics platforms; Collaborate with stakeholders to deliver data-driven solutions; Monitor and optimize pipeline performance and storage costs; Create technical documentation; Implement data security best practices; Leverage AI tools to enhance workflows.

Seniority

Senior, hands-on IC