Audio QA Lead - Part Time Contractor

About Besimple AI

Voice data for AI

Tech

# Technology & Hard Problems ## **Product Surface** Besimple generates **task-specific annotation interfaces and guidelines on the fly**, runs **human-in-the-loop (HITL) workflows** at scale, and trains **AI judges** that learn from human decisions to triage easy cases and flag ambiguous ones. We support **multimodal data** (text, chat, audio, video, traces) and **enterprise needs** like **on-prem deployment** and **fine-grained access control**. Under the hood, we optimize for **latency**, **correctness**, and **adaptability**—simultaneously. ## **Hard Technical Problems We’re Tackling** * **Generative UI for Any Data Shape** Turn arbitrary inputs—**JSON logs, multi-turn dialogs, code diffs, speech transcripts, video frames**—into ergonomic, versioned UIs with validation and assistive affordances (schema inference, promptable components, live preview with safe defaults). * **Human-in-the-Loop Orchestration** Route tasks to the right experts, enforce **calibration and quality gates**, measure **IRR**, and run **adjudication** when disagreement is informative—not noise. * **AI-Judge Training & Control** Distill human rubrics into model-based evaluators that **score live traffic**, self-update with new human decisions, and stay inside **guardrails** (confidence thresholds, policy constraints, auditability). * **Production-Grade Eval** Build **gating suites and regression tests** aligned to product KPIs and safety constraints; snapshot datasets; track drift; and **plumb production signals** back into evaluation and training. * **Enterprise Delivery** **On-prem optional installs**, isolation-by-tenant, **SSO/RBAC**, and **audit trails** that satisfy infosec without slowing iteration. ## **What You’ll Own** End-to-end slices of the product—e.g., building a new **multimodal interface**, designing a **calibration workflow** that improves IRR, shipping a **rubric-aware AI judge** for a new domain, or tightening **dataset lineage** so a customer can trace a production decision back to ground truth. ## **Why This Is a Great Fit for Builders** This work sits at the intersection of **product engineering, systems design, and applied AI**. You’ll ship tangible interfaces, shape **evaluation science**, and see your work **block real regressions**. The feedback loop is measured in **better models in production**, not vanity benchmarks.

The role

### About the roleWe are hiring an Audio QA Lead to support the development of high-quality training datasets for next-generation voice AI models.In this role, you will work hands-on to improve the quality, consistency, and usability of speech datasets across applications such as text-to-speech, transcription, speech-to-speech, ASR, and conversational voice systems. Your work will directly influence how data is collected, reviewed, and delivered for real-world model training.You will work across three core areas: defining and applying audio quality standards, recording high-quality speech on demand, and performing annotation and QA across speech datasets. This is not a generic audio production role. The work focuses on making audio usable for model training and requires a strong understanding of how data quality impacts model.**This is a part-time contractor role that can turn into full-time role.** ## What you'll do* Develop, refine, and apply audio quality guidelines for speech and voice datasets.* Review audio files against technical, linguistic, and task-specific standards, making clear approval, rejection, or revision decisions.* Identify audio and annotation issues such as background noise, clipping, distortion, plosives, echo, low signal, segmentation errors, transcript mismatches, and speaker-label inconsistencies.* Perform annotation and QA tasks, including transcription, timestamp validation, VAD/segmentation, diarization, pronunciation checks, and metadata review.* Record speech based on provided scripts and performance guidelines, delivering natural, high-quality, specification-compliant audio.* Document edge cases, update review rubrics, and improve internal SOPs and quality standards.* Collaborate with research, ML, and operations teams to translate model requirements into data specifications and evaluation criteria.* Ensure consistency and integrity across audio files, transcripts, annotations, and associated metadata.## Who we're looking forThe ideal candidate has direct experience working with audio AI datasets and understands what makes speech data effective for model training. You have a strong ear for audio quality, are comfortable applying annotation standards, and can consistently produce and evaluate high-quality recordings.* Direct experience working with audio AI training datasets or evaluation workflows.* Hands-on experience with TTS, ASR, transcription, speech-to-speech, or related voice AI systems.* Experience developing or applying audio quality standards in production environments.* Experience with speech annotation tasks such as transcription, timestamp QA, VAD/segmentation, and diarization.* Strong auditory judgment with the ability to consistently identify subtle audio quality issues.* Ability to produce high-quality recordings in a controlled, quiet environment using professional or near-professional equipment.* Strong written communication skills with the ability to provide clear, actionable feedback.* High attention to detail and sound judgment when evaluating edge cases.* Comfort working with structured data formats such as spreadsheets, CSV, or JSON.### Bonus qualifications* Experience with audio tools such as Audacity, Praat, or similar.* Basic scripting skills in Python, Bash, or SQL for QA or dataset analysis.* Background in linguistics, phonetics, speech research, or voiceover work.* Experience evaluating both real and synthetic audio.* Multilingual experience or familiarity with accents and dialect variation.* Familiarity with compliant handling of consented and licensed voice data.