Senior Data Engineer
Core
Designing, building, and maintaining scalable data pipelines and infrastructure on AWS to support analytics, machine learning, and Generative AI initiatives.
Role type
Senior IC data engineer (cloud & AI infrastructure)
Builds
End-to-end data pipelines, data integration processes, and cloud-based data architectures
Domain
Cloud data engineering, Generative AI, and ML infrastructure
Deliverable
production ML models
Required skills
AWS cloud services (S3, Glue, Lambda, Redshift), Databricks, Apache Spark, SQL, Apache Airflow, data modeling, ETL/ELT frameworks, CI/CD pipelines, vector databases, embeddings, RAG architectures, LLM APIs, LangChain, MLOps workflows (via careerplan.io/jobs/ef5d3a52-4fd8-4263-89d9-63f566478bf6-senior-data-engineer-at-tiger-analytics-inc)
Preferred skills
Experience with LLM-based applications, knowledge retrieval systems, legacy system migration
Technologies
AWS, Databricks, Apache Spark, Apache Airflow, Amazon Redshift, Amazon S3, AWS Glue, AWS Lambda, Git, LangChain
Responsibilities
Design, develop, and deploy end-to-end data pipelines on AWS; Implement data processing and transformation workflows; Build and maintain orchestration workflows using Apache Airflow; Support data preparation and ingestion for AI/ML and Generative AI workloads; Enable data pipelines for LLM-based applications, vector embeddings, and knowledge retrieval systems; Lead the migration of legacy data systems to modern cloud-based data architectures; Develop and maintain CI/CD pipelines for data workflows; Collaborate with data scientists, ML engineers, and AI teams to ensure data availability; Optimize data pipelines for performance, reliability, scalability, and cost-effectiveness
Seniority
Senior, hands-on IC
