CareerPlanGet AI match score →

Principal AI Systems Engineer

🌐 Remote💼 Full-time💰 $175,000–$175,000🗓 2026-06-01 → 2026-07-31

Core

Design, build, validate, and deploy production AI systems (LLMs, RAG, agents) to improve operational productivity in a regulated life sciences environment.

Role type

Principal AI Systems Engineer (hands-on IC)

Builds

Production-grade AI systems, RAG pipelines, AI agents, and integrations for internal business processes.

Domain

Life Sciences / Biotechnology / Regulated Manufacturing

Deliverable

production ML models

Required skills

LLM application development, RAG system architecture, Python, AWS (S3, Redshift, Bedrock), MLOps, CI/CD, vector storage, prompt engineering, AI security controls, agent design, evaluation harnesses

Preferred skills

Model Context Protocol (MCP), enterprise search (OpenSearch/Elasticsearch), US regulatory knowledge (21 CFR Part 11, GxP, HIPAA)

Technologies

Python, AWS, S3, Redshift, Bedrock, OpenSearch, Elasticsearch, Git, Docker

Responsibilities

Design and ship AI systems end-to-end; implement production RAG systems; build evaluation harnesses; integrate AI with enterprise systems; maintain LLM security controls; design and deploy AI agents; establish engineering practices; evaluate AI vendors; implement technical controls; author technical documentation; mentor junior engineers

Seniority

Principal, hands-on IC

Rewrite
## Overview Iovance Biotherapeutics aims to be the global leader in innovating, developing and delivering tumor infiltrating lymphocyte (TIL) therapy for people with cancer. We are pioneering a transformational approach to treating cancer by harnessing the ability of the human immune system to recognize and attack diverse cancer cells in each patient. The Iovance TIL platform has demonstrated promising clinical data across multiple solid tumors. We are committed to continuous innovation in cell therapy, including gene-edited cell therapy, which may be a promising option for patients with cancer. ## Responsibilities - Design, build, and ship AI systems against the approved Iovance AI roadmap, including end-to-end ownership of architecture, retrieval pipelines, prompts, evaluation, integrations, and deployment for assigned use cases. - Implement production-grade Retrieval-Augmented Generation (RAG) systems on Iovance’s AWS infrastructure (S3, Redshift, Bedrock), including chunking strategies, embedding selection, vector storage, retrieval and reranking, grounding, and citation handling appropriate to high-accuracy use cases. - Build, maintain, and run evaluation harnesses for AI systems, including held-out test sets, accuracy and grounding metrics, hallucination detection, adversarial inputs, and regression testing across model and prompt changes; treat evaluation as a first-class engineering deliverable, not an afterthought. - Design and implement integrations between AI systems and Iovance enterprise systems using Model Context Protocol (MCP), APIs, and event-driven patterns, applying least-privilege access principles and partnering with IT Security on integration approval. - Own and maintain LLM security controls for production AI systems, including input and output guardrails, prompt injection and jailbreak defenses, sensitive data redaction (PII, PHI, Iovance Confidential Information and Intellectual Property), content moderation, and abuse monitoring, working in partnership with IT Security. - Design, develop, deploy, and maintain AI agents (multi-step reasoning systems that combine LLMs with tools, retrieval, and planning) appropriate for use in a regulated life sciences environment, including bounded scope, defined human oversight, traceability of agent decisions, and safe handling of write-back actions to systems of record. - Establish and uphold modern engineering practices for AI development including version control for code, prompts, and evaluation sets; CI/CD pipelines; environment separation (dev, test, production); and reproducible builds. - Conduct hands-on technical evaluation of AI vendors, tools, and Foundation Models when build-vs-buy decisions are under consideration; produce concise, fact-based recommendations to the IT function lead and AI Governance Committee, including proof-of-concept results where appropriate. - Implement and operate technical controls for production AI systems including audit logging, access management, prompt and model change control, model registry, ongoing performance monitoring, and incident detection, in alignment with Iovance policies. - Author technical documentation appropriate to the system risk tier, including architecture diagrams, data flow diagrams, evaluation reports, runbooks, and validation deliverables; contribute to the Iovance AI Validation Playbook. - Mentor junior engineers who collaborate on AI projects; stay current on rapid advances in AI tooling, models, and engineering best practices, and bring technical recommendations forward. ## Requirements - 10+ years of progressive software and/or AI/ML engineering experience, with the most recent 1+ years dedicated primarily to LLM-based application development. Demonstrated track record of shipping production AI systems that real users depend on. - Deep, hands-on production experience with Retrieval-Augmented Generation (RAG) including chunking strategies, embedding models, vector stores, retrieval and reranking, and grounding for high-accuracy use cases. Working production experience with Model Context Protocol (MCP) or equivalent agent/tool integration patterns. - Practical working experience across multiple frontier model families (e.g., Anthropic Claude, OpenAI GPT, Google Gemini, leading open-weight models), with the judgment to select among them based on task fit, accuracy, cost, latency, safety properties, and data-handling commitments. Awareness of model capability changes and how to evaluate new model releases. - Strong software engineering fundamentals: production Python (and ideally one other language); modern Agile delivery; version control (Git); CI/CD; containerization; cloud platforms (preferably AWS, with working familiarity). ## Nice to Have - Experience with regulated environments and compliance requirements in life sciences. - Familiarity with clinical data handling and privacy regulations. - Experience with AI governance frameworks and validation processes. - Strong communication and mentoring skills. ## Benefits - Opportunity to work on cutting-edge AI systems in a regulated life sciences environment. - Collaborative and innovative work culture. - Professional growth and mentorship opportunities.
Sourced via greenhouse · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.
Apply on Greenhouse ↗