Data Engineer
Core
Design, build, and scale a modern cloud-native Data Platform supporting batch, streaming, analytics, and real-time data use cases.
Role type
Senior IC Data Engineer (Data Platform)
Builds
Cloud-native data platforms, reusable ingestion/transformation frameworks, streaming platforms, and data lake/lakehouse foundations.
Domain
Cloud infrastructure, distributed data systems, real-time event-driven architectures.
Deliverable
production ML models | product features | infrastructure
Required skills
Apache Spark, Apache Flink, SQL, Python, distributed data processing, Apache Kafka, AWS Glue, AWS Athena, AWS DMS, Amazon S3, Amazon CloudWatch, IAM, networking, data partitioning, CDC architecture, schema evolution, high availability, fault tolerance, data governance.
Preferred skills
Apache Airflow, dbt, Delta Lake, Apache Iceberg, Docker, Kubernetes, Terraform, CI/CD pipelines.
Technologies
Apache Spark, Apache Flink, Apache Kafka, AWS Glue, AWS Athena, AWS DMS, Amazon S3, Amazon CloudWatch, Apache Airflow, dbt, Delta Lake, Apache Iceberg, Docker, Kubernetes, Terraform.
Responsibilities
Design scalable enterprise data platform architecture on AWS; Build reusable frameworks for data ingestion, transformation, orchestration, and consumption; Develop batch and real-time data pipelines using distributed processing technologies; Build and maintain streaming platforms for event-driven architectures; Create standardized data lake/lakehouse foundations; Implement metadata management, schema evolution, lineage, governance, and access control; Develop platform capabilities for data reliability, observability, monitoring, and alerting; Enable self-service data access and scalable data consumption layers; Build CDC and incremental ingestion frameworks; Optimize platform scalability, performance, fault tolerance, and cloud cost; Automate deployments, infrastructure provisioning, and CI/CD for data workloads; Collaborate with Analytics, Product, and Engineering teams to onboard new data domains; Ensure platform security, compliance, and operational excellence.
Seniority
Senior, hands-on IC