AI Engineer, Post-Training & Evaluation

About Coval

Simulation & Evaluation that scales voice and chat AI agents

Tech

NextJS, Typescript, Python

The role

## **The role**Own the models behind Coval’s judges. Coval is the evaluation platform for conversational AI: we help enterprise teams simulate, monitor and improve voice and chat agents before those agents ever reach a customer. Because our product is the judge, this role sits at the center of the company.You’ll own the model side end to end. You’ll design and ship the LLM-as-a-judge evaluations that score voice and chat conversations at scale, then sharpen them with LoRA fine-tunes on open-weight models and non-tuning work like tool use and context engineering. Your improvements ship directly into the product enterprises rely on to launch dependable voice agents.## **What you’ll do*** Design and ship the LLM judges that score conversations: rubrics, judge prompting, and calibration against human raters* Build the reliability tracking that keeps judges honest from release to release* Run post-training where training beats prompting: LoRA adapters on open-weight models, trained on data distilled from our own scored conversations* Everywhere else, do the non-tuning capability work: tool use, structured outputs, and context engineering* Keep asking which lever actually moves the metric, and focus there## **What we’re looking for*** 4+ years as an engineer, with strong systems instincts* A track record of shipping measurable quality gains by pairing rigorous measurement with targeted training, not vibes and prompt tweaks* You’ve built eval infrastructure you actually trust* You’ve run LoRA/SFT experiments end to end, including preparing your own data rather than relying on clean data from other teams, so you know which lever really wins* High agency: on a small Series A team, nobody hands you tickets. You figure out what matters next and drive it there* Ambitious and kind. You’ll be part of the group that founds our engineering org, sets its bar and grows the team