Principal Software Engineer
Core
Architect and lead the design, development, and maintenance of scalable data engineering infrastructure, including migrating on-prem applications to cloud-based PaaS and building end-to-end data solutions.
Role type
Principal Software Engineer (Data Engineering & Architecture)
Builds
Scalable data pipelines for batch and real-time processing, cloud-based data solutions, and data-intensive applications with APIs and streaming pipelines.
Domain
Data Engineering, Cloud Computing, Big Data
Deliverable
production ML models
Required skills
Data architecture, data modelling (dimensional, relational, NoSQL), ETL/ELT design, big data technologies, cloud platforms, GenAI integration, database design and optimization, data orchestration, data security and governance
Preferred skills
Microservices architecture, API design, data mesh/data fabric architectures, data lake and Lakehouse implementation
Technologies
Python, SQL, Java/Scala, Apache Spark, Hadoop, Kafka, Hive, Trino, AWS (S3, EC2, RDS, Lambda, DynamoDB, Glue), Azure (Data Factory, Synapse, Databricks), GCP (Big Query, Dataflow, Dataproc), Snowflake, Redshift, Big Query, Fabric Data Warehouse, Airflow, dbt, Informatica, Talend, Glue, PostgreSQL, MySQL, SQL Server, Oracle, MongoDB, Cassandra, DynamoDB, Influx DB, Timescale DB, Docker, Kubernetes, Git, GitLab/GitHub, Jenkins, GitHub Actions, GitLab CI
Responsibilities
Drive innovations by keeping current on emerging cloud technologies; Act as a Subject Matter Expert for migrating on-prem applications to cloud-based PaaS; Architect end-to-end data solutions; Provide guidance in the migration and transfer of multi-terabyte databases; Design solutions to prepare and transform data into a usable state; Lead the design, development, and maintenance of data engineering infrastructure; Mentor and provide technical guidance to junior data engineers; Collaborate with data scientists, analysts, and business stakeholders to understand data requirements; Optimize data models and database designs for performance and reliability; Implement data quality controls and monitoring systems; Evaluate and recommend new data technologies and tools; Deploy and manage cloud-based data solutions; Ensure data security, compliance, and governance standards are met
Seniority
Principal, hands-on IC with technical leadership