Software Engineering Advisors- Hybrid
Core
Design and maintain scalable data pipelines in Databricks to deliver high-quality, timely data for client deliverables in the healthcare sector.
Role type
Senior Data Engineering Advisor
Builds
Enterprise-level data solutions, consolidated data warehouses, and data lakes supporting strategic decision-making
Domain
Healthcare / Big Data Engineering
Deliverable
production ML models | product features | dashboards & analysis
Required skills
PySpark, SQL, Scala, Shell scripting, CI/CD pipelines, Databricks Delta tables, AWS Glue, Terraform, NLP techniques, statistical analysis
Preferred skills
Experience with PostgreSQL, MySQL, SQL Server, Cassandra, Snowflake, Hadoop, Hive, Kafka, RabbitMQ, Tableau, Power BI, GitHub Actions, GitLab, Jira
Technologies
Databricks, PySpark, Scala, SQL, Shell, AWS (Glue, IAM, EC2, EMR, RDS, S3, Athena, Step Functions, Lambda), GitHub Actions, Terraform, Tableau, Power BI, Kafka, RabbitMQ
Responsibilities
Design and implement modular ETL workflows for data ingestion, cleansing, and transformation; Monitor production jobs and troubleshoot data failures; Develop consolidated enterprise data warehouse and data lake; Optimize data models and transformation logic; Apply statistical techniques to interpret healthcare data; Develop and manage CI/CD pipelines; Ensure data compliance with HIPAA and PHI guidelines; Translate business requirements into technical solutions
Seniority
Senior, hands-on IC