Big Data Engineer_BST
Core
Design, build, and maintain scalable data pipelines and analytics workflows to process large-scale test and experimental data, transforming raw testing data into high-quality datasets for operational monitoring and decision making.
Role type
Senior Big Data Engineer
Builds
Scalable data pipelines, reliable data models, and curated datasets for downstream analytics and visualization.
Domain
Automotive/Industrial testing data processing
Deliverable
production ML models | product features | dashboards & analysis
Required skills
Python, PySpark, Spark architecture, distributed data processing, ETL/ELT design, SQL, data modeling, data validation, query optimization
Preferred skills
Tableau, data warehouse concepts
Technologies
PySpark, Spark SQL, Tableau
Responsibilities
Design and maintain scalable data pipelines; process and transform large volumes of structured and semi-structured test data; implement efficient ETL/ELT workflows; optimize Spark jobs for performance and cost; design data models and curated datasets; implement data validation and quality checks; prepare analysis-ready datasets for Tableau dashboards.
Seniority
Senior, hands-on IC
