Data/AI Engineer (Databricks, AWS, Python, AI))
Core
Design, develop, and support scalable cloud-based data pipelines, data products, and AI-driven applications for investment management and client services.
Role type
Senior Data/AI Engineer (Cloud & GenAI)
Builds
Scalable batch and real-time data pipelines, data products, and cloud-based data solutions on AWS and Databricks.
Domain
Financial Services / Data Engineering / Generative AI
Deliverable
production ML models | product features (via careerplan.io/jobs/180496-dataai-engineer-databricks-aws-python-ai-at-vanguard)
Required skills
Python, PySpark, Advanced SQL, AWS data services (S3, Glue, Lambda, Redshift, Kinesis), Databricks, Airflow, Kafka, Terraform, Docker, CI/CD, RAG, Prompt Engineering, Bedrock, LangChain
Preferred skills
Generative AI workflows, Agentic workflows, Tool/function calling, OpenSearch Serverless vectors, CDK/CloudFormation
Responsibilities
Build and optimize data ingestion, transformation, and storage solutions; Design and maintain scalable data pipelines; Implement data quality controls and observability; Troubleshoot production issues and perform root-cause analysis; Contribute to AI-assisted development practices and reusable components.
Seniority
Senior, hands-on IC
