Técnico Databricks / Hadoop
Core
Administer and operate Big Data platforms including Azure Databricks and Hadoop ecosystems to support data engineering and science teams.
Role type
Senior Data Infrastructure Engineer (Databricks/Hadoop)
Builds
Distributed data clusters, data pipelines, and cloud migration projects
Domain
Cloud Data Infrastructure (Azure, Hadoop, Spark)
Deliverable
infrastructure
Required skills
Azure Databricks, Hadoop ecosystem (HDFS, Hive, Spark, YARN), PySpark/Scala, Infrastructure as Code (Terraform, ARM), Data Governance (IAM, RBAC), Cloud environments (Azure/AWS), DevOps/DataOps practices
Preferred skills
Data Lake/Lakehouse architectures, large-scale critical data pipelines, cloud migration experience, data monitoring and performance tools
Technologies
Azure Databricks, HDFS, Hive, Spark, YARN, PySpark, Scala, Terraform, ARM, Azure Data Factory, Airflow, Git
Responsibilities
Administer and operate Big Data platforms; Manage and optimize distributed clusters; Monitor and troubleshoot large-scale data pipelines; Automate deployments using IaC; Ensure data security and governance; Provide technical support to data teams; Act on on-premises to cloud migration projects