TC
SDET - AI/ML
TechDigital Corporation
🌏 Worldwide
Remote
10 months ago
- Azure DevOps
- Python
- pytest
- Test Automation
- AI/ML
- RAG
- MLflow
- MLOps
- CI/CD
- Databricks
- Azure Cloud
- YAML
- Cucumber
- BDD
- Integration Testing
- Microservices
- Performance Testing
- JMeter
- PowerShell
- Agile
- Scrum
- Jira
10 months ago
Must have skills: SDET skills, Azure devops, python, pytest test automation and Framework, AI ML practices, RAGAS, LLMs
Brief JD:
- Experience in designing LLM/RAG test automation solutions
- Experience in testing for bias, drift, and fairness.
- Familiarity with performance metrics (precision, recall, F1, ROC-AUC).
- Knowledge of MLflow MLOps framework
- Knowledge of tools for synthetic data generation and boundary testing
- Knowledge of Azure DevOps for CI/CD pipeline development
- Experience in RAG/pipeline evaluation frameworks -
- pytest: Test automation framework
- DeepEval or TruLens: LLM test assertions
- RAGAS: RAG-specific metrics
- Eleuther: LLM evaluation harness
- Garak or Promptfoo: LLM red-teaming
- Evidently: Drift/performance monitoring
- Exposure to explainability frameworks (SHAP, LIME, Captum)
• Practical knowledge of Databricks, Azure Cloud services land distributed data validation.
• Proficiency with Azure DevOps pipelines YAML templates, agent pools, CICD workflows.
• Experience in implementing AIML practices in testing e.g., test generation, anomaly detection, log analysis to improve test efficiency and coverage.
• Familiarity with Cucumber (BDD) and test reporting frameworks (e.g., Allure).
• Strong understanding of integration testing across Databricks streaming jobs, applications and microservices.
• Experience with performance testing tools (JMeter, LoadRunner, or equivalent).
• Experience with Pester framework for validating PowerShell scripts and infrastructure automation.
• Solid understanding of Agile/Scrum methodology and handson usage of JIRA.
• Excellent problem-solving, debugging, and communication skills, with the ability to advise development teams on testing best practices.
SDET - AI/ML · TechDigital Corporation