Big Data Engineer
Core
Design, build, and optimize scalable data pipelines for large-scale data lake ingestion, handling hundreds of TBs.
Role type
Senior Big Data Engineer
Builds
Scalable data pipelines and lakehouse architectures for enterprise data management
Domain
Enterprise data security and management (SaaS)
Deliverable
production ML models | product features
Required skills
Apache Spark, Delta Lake, Apache Iceberg, Python, Java, SQL, AWS (S3, Glue, Athena, Lambda), Databricks, Snowflake, AWS Redshift, IaC, observability
Preferred skills
Cost optimization, data governance, security-by-design, data modeling
Technologies
Spark, Delta Lake, Apache Iceberg, Databricks, Snowflake, AWS Redshift, AWS S3, AWS Glue, AWS Athena, AWS Lambda
Responsibilities
Design and monitor Spark data pipelines for large-volume ingestion; Optimize performance, reliability, and costs of big data infrastructure; Manage and optimize lakehouse table formats; Apply security and compliance best practices across storage and compute layers
Seniority
Senior, hands-on IC
