ES
AI EVAL/QE
Epsilon Solutions LTD
- Location not stated
- Hybrid
- 4 hours ago
- AI
- Machine Learning
- AI/ML
- Python
- Test Automation
- Integration Testing
- GitHub
- GitHub Actions
- Claude Code
- Copilot Studio
- Foundry
- LangSmith
- Dynatrace
4 hours ago
TORONTO - HYBRID - FULLTIME
10+ YRS EXP.
PRIMARY SKILLS:Gen AI Automation Testing, AI evaluator/ quality engineer
Define and execute the evaluation approach for generative-AI solutions, proving that models, agents and AI-assisted workflows are accurate, safe, reliable, performant and fit for production under TD controls. Requires 10+ years in quality engineering and at least 2 years evaluating AI/ML systems, with strong Python, test automation, API and integration testing, prompt and response test-set design, groundedness/relevance and hallucination assessment, regression, bias/safety, performance, penetration and non-functional testing; candidates should be comfortable with GitHub/GitHub Actions, GitHub Copilot, Claude Code, Copilot Studio, Foundry or LLM Gateway patterns, LangSmith, AppInsights, Log Analytics and Dynatrace, and must produce auditable evidence, defect analytics and go/no-go recommendations for architecture, privacy, cyber and risk reviewers.
AI EVAL/QE · Epsilon Solutions LTD