CareerPlanGet AI match score →

Founding Engineer — Open-Source AI Deployment Tools

Onsite or remote • San Francisco+3💼 Full-time🗓 2026-06-25

Core

Building developer tools and backend infrastructure to deploy, serve, monitor, and improve open-source AI models in real-world environments.

Role type

Founding Engineer (Open-Source AI Deployment)

Builds

Developer tools for model deployment, serving, monitoring, and improvement; backend systems and APIs for AI workflows.

Domain

Open-source AI, Model Serving, Infrastructure

Deliverable

production ML models | product features | infrastructure

Required skills

vLLM, TGI, Ollama, llama.cpp, TensorRT-LLM, Hugging Face, Meta Llama, Mistral, Qwen, DeepSeek, vector search, observability, dataset management, API design, backend system architecture, rapid prototyping

Preferred skills

Experience with open model ecosystems, familiarity with fine-tuning and evaluation workflows

Technologies

vLLM, TGI, Ollama, llama.cpp, TensorRT-LLM, Hugging Face, Meta Llama, Mistral, Qwen, DeepSeek

Responsibilities

Build developer tools for deploying and serving open-source AI models; Design workflows for fine-tuning, evaluation, and benchmarking; Integrate with open-source AI ecosystems for inference and observability; Build reliable backend systems and APIs; Contribute to early technical decisions and product direction.

Seniority

Founding, hands-on IC with strategic input

Rewrite
## About the role This role is for someone who understands how open models are served, fine-tuned, evaluated, and deployed in real-world environments — but you do not need to be an expert in every area. We care more about strong engineering ability, curiosity, and the willingness to learn quickly. ## Responsibilities - Build developer tools that make it easier to deploy, serve, monitor, and improve open-source AI models. - Work with modern model-serving technologies such as vLLM, TGI, Ollama, llama.cpp, TensorRT-LLM, or similar systems. - Help design workflows for fine-tuning, evaluation, benchmarking, and model comparison. - Integrate with popular open-source AI ecosystems, including tools for inference, vector search, observability, datasets, and model hosting. - Experiment with open models from ecosystems such as Hugging Face, Meta Llama, Mistral, Qwen, DeepSeek, and others. - Build reliable backend systems, APIs, and infrastructure that make complex AI workflows feel simple for users. - Work closely with users to understand pain points around deploying and improving open models. - Move quickly from idea to prototype to product, especially in areas that are still evolving. - Contribute to early technical decisions, product direction, and engineering culture as part of the founding team.
Sourced via wellfound · Listed on CareerPlan, which tracks 70,000+ jobs from 20+ sources.
Apply on Wellfound ↗