Senior/Staff ML Ops Engineer

Role Summary:

As a Senior MLOps Engineer at Filevine, you'll sit at the intersection of machine learning, platform infrastructure, and product velocity. You'll build and own the systems that make Filevine's AI capabilities faster to develop, safer to ship, and easier to trust, at scale.

You will be responsible for the full stack of ML infrastructure: evals, observability, model serving, annotation tooling, and the prompt platform that lets every team move with confidence.

Responsibilities

• Setup and maintain LLM observability frameworks/tools

• Help improve data annotation tooling

• Ensure stability of LLM calls (rate limits, provisioned throughput, backups, …)

• Help to drive security review processes for AI vendors and providers 

• LLM cost optimization recommendations (caching, batching, identification of workflow parts causing high costs, etc.)

• Hosting finetuned/open weight machine learning models

• Helping with LLM evaluations (tooling/framework) with the current main focus on agentic evals

• Platform tooling for enabling non-technical people (e.g. PMs) to iterate on prompts

Qualifications

• 5+ years building and operating software systems end-to-end

• Hands-on experience with ML infrastructure: model serving, training pipelines, or LLM integrations in production

• Strong understanding of cloud infrastructure and distributed systems (primarily AWS)

• Familiarity with observability tooling and cost management for LLM workloads

• Experience with or openness to: Python, Kubernetes, Terraform

• Thrives in a remote-first, async environment: clear communicator, high ownership, low ego

• Bonus: experience with eval frameworks, annotation tooling, or prompt management platforms