Ai Bioinformatics Benchmarking Engineer

## About the Role At Sequencing.com, we are building AI systems that interpret real human genetic data at scale. Accuracy is foundational to everything we do. We are hiring an AI Bioinformatics Benchmarking Engineer to build and operate the evaluation and validation infrastructure that ensures our AI systems are correct, reliable, and regression-protected. This role is execution-focused. You will design, implement, and maintain the benchmarking systems that continuously test AI outputs against curated genomic datasets. Location: Remote. This position is not open to candidates residing in New York City. ## What You'll Own - Automated benchmarking pipelines for AI genetic interpretation - Variant-level and case-level regression datasets - Accuracy metrics and performance tracking dashboards - Failure detection and degradation alerts - Continuous validation of AI outputs as prompts, agents, and models evolve You will work closely with the AI Bioinformatics Engineering Lead to operationalize interpretation standards into measurable systems. ## Key Responsibilities ### Benchmarking & Evaluation Infrastructure - Design and implement automated evaluation pipelines for AI outputs. - Build regression datasets covering variant normalization, transcript ambiguity, and disease-level mapping. - Execute large-scale validation runs across curated VCF datasets. - Track performance metrics over time and flag accuracy degradation. ### Dataset Curation & Edge Case Coverage - Build and expand a structured question bank using DV3 and related datasets. - Identify and encode edge cases for systematic testing. - Maintain ground truth datasets for pathogenic and benign variants. - Ensure benchmarking reflects real-world genomic complexity. ### Continuous Validation & Monitoring - Implement automated regression tests for AI releases. - Create dashboards for tracking variant-level and case-level accuracy. - Partner with engineering to integrate validation into CI/CD workflows. - Diagnose and document failure modes. ## What We're Looking For - Degree in Bioinformatics, Computational Biology, Genetics, or related field. - Strong experience working with VCFs and real-world genomic datasets. - Familiarity with ClinVar, dbSNP, HGVS standards, and transcript mapping. - Experience building evaluation frameworks or testing pipelines such as Langfuse - Detail-oriented mindset with systems thinking. - Experience evaluating LLM-based systems. - Experience building automated QA systems. - Familiarity with data engineering workflows. ## About Sequencing Sequencing is powering the future of personalized health. We are the platform that transforms a person's genome into a lifelong source of straightforward, actionable insights - for individuals, families, and healthcare professionals. By providing the application layer to your DNA, we turn complex genetic data into recurring value throughout every stage of life. Our unique, clinical-grade Whole Genome Sequencing unlocks the most complete view of your DNA, screening for almost all known health conditions, traits, and medication responses - and translating that information into clear, actionable steps for managing health risks, optimizing wellness, and extending longevity. As whole genome sequencing becomes a standard part of life, Sequencing is already the trusted platform where people safely store their DNA data - and will become the lifelong home for every genome on Earth. We're a fully remote, global team of innovators, scientists, and builders, united by a mission to make genomics accessible, actionable, and life-changing for everyone. Proudly named GenomeWeb's Best Place to Work in 2024, Sequencing is venture-backed and scaling rapidly to meet surging demand in the fast-growing personal genomics market. If you're excited to help redefine how the world thinks about health, join us in building the platform that brings every genome to life. Sequencing Inc. is an equal opportunity employer. All aspects of employment, including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate based on race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, veteran status, or any other status protected under federal, state, or local law. Sequencing Inc. may use artificial intelligence (AI) or automated tools as part of the recruiting and hiring process, including for application screening and candidate evaluation. These tools may analyze information provided in your application materials or during the hiring process. Where required by applicable law, candidates may request a reasonable accommodation or an alternative selection process by contacting [email protected].