AI QA/Testing Lead

About Rimini Street, Inc.

Rimini Street, Inc. (Nasdaq: RMNI), a Russell 2000® Company, is a proven, trusted global provider of end-to-end, mission-critical enterprise software support, managed services and innovative Agentic AI ERP solutions, and is the leading third-party support provider for Oracle, SAP and VMware software.

Our comprehensive portfolio of unified solutions help run, manage, support, customize, configure, connect, protect, monitor, and optimize enterprise application, database and technology software, enabling our clients to achieve better business outcomes, significantly reduce costs and reallocate resources towards strategic projects.

The Company has signed thousands of contracts with Fortune Global 100, Fortune 500, midmarket, public sector and government organizations who selected Rimini Street as their trusted, proven mission-critical enterprise software solutions provider and achieved better operational outcomes, realized billions of US dollars in savings and funded AI and other innovation investments.

We are seeking a QA Testing Lead - Agentic ERP Platform. This hybrid role is based in our Selangor or Penang office.

Position Summary

TheQA Testing Lead owns the quality assurance strategy for Rimini Street’s Agentic ERP Platform, establishing the testing frameworks, standards, and processes required to validate AI-driven systems in production. This role defines how the team ensures platform reliability, accuracy, and safety across non-deterministic agent behaviors—with a particular focus on proving the platform is trustworthy enough for SOX-regulated ERP environments.

Reporting to the VP, Platform Engineering, this leader builds the QA function from the ground up—hiring and mentoring QA engineers, defining AI evaluation strategies with measurable accuracy thresholds, and establishing quality gates that balance rapid iteration with enterprise-grade reliability. The ideal candidate combines deep QA leadership experience with a strong understanding of how to test AI and ML-powered systems.

Essential Duties & Responsibilities

AI Evaluation Strategy & Quality Gates

• Own the AI evaluation strategy: define evaluation rubrics, accuracy thresholds, regression baselines, and gating criteria for agent behaviours.
• Establish testing frameworks and methodologies suited to non-deterministic AI agent outputs, including statistical validation approaches and LLM-as-judge evaluation patterns.
• Design quality gates for the CI/CD pipeline that enforce standards without blocking delivery velocity.
• Define and maintain test plans, test cases, and traceability matrices aligned to platform requirements.
• Own SOX-grade audit evidence validation—proving that every agent action produces a complete, immutable audit chain suitable for financial-controls-regulated client environments.

Team Leadership

• Build and lead the QA engineering team for the Malaysia engineering team.
• Mentor QA engineers on testing methodologies for AI-powered systems.
• Establish team processes, code review standards for test automation, and knowledge sharing practices.
• Coordinate testing activities across distributed engineering teams in India, Malaysia, and Brazil.

Test Automation & Infrastructure

• Drive test automation strategy, identifying high-value areas for automated coverage.
• Oversee the development of automated test suites for API, UI, and end-to-end testing.
• Design air-gap test environment strategy for validating releases destined for disconnected client environments where standard cloud test infrastructure does not apply.
• Establish and maintain test environments that replicate production configurations.
• Implement test data management strategies for complex enterprise scenarios.

Quality Assurance Operations

• Own defect management processes, including triage, prioritization, and resolution tracking.
• Produce quality metrics and dashboards that provide visibility into platform health.
• Conduct root cause analysis on production issues and drive systemic improvements.
• Coordinate release validation activities and sign-off processes.
• Coordinate chaos engineering activities: failure injection on durable workflows, network partition simulation, and replay verification.
• Coordinate with platform engineering teams to validate that delivery teams use platform primitives correctly and adhere to established patterns.

Experience

• 8+ years of QA engineering experience, with at least 3 years in a lead or management role.
• Proven experience building QA teams and establishing testing processes from scratch.
• Experience testing AI/ML systems, APIs, or complex distributed platforms.
• Track record of implementing test automation frameworks at enterprise scale.
• Experience with agile/scrum development environments.
• Background in enterprise software, ERP systems, or B2B platforms preferred.

​Technical Skills

Required

• Test automation frameworks: Pytest, Selenium, Playwright, or Cypress.
• API testing: Postman, REST Assured, or equivalent tools.
• API contract testing: Pact or equivalent for schema validation across service boundaries.
• AI evaluation tooling: LLM-as-judge frameworks, Promptfoo, DeepEval, or equivalent for agent output regression testing.
• Python or JavaScript/TypeScript for test scripting and automation.
• CI/CD integration: GitHub Actions, Jenkins, or similar.
• Performance testing: JMeter, Locust, k6, or similar.
• Chaos engineering: failure injection, network partition simulation, and workflow replay verification for distributed systems.
• Test management and defect tracking tools (Jira, TestRail, or equivalent).
• Understanding of AI/ML system testing: non-deterministic output validation, evaluation metrics, and data-driven testing.
• Git version control and branch management strategies.

Preferred

• Experience with Angular application testing (Angular 14+).
• OpenTelemetry and observability-driven testing (trace inspection, distributed context propagation).
• Familiarity with Snowflake testing patterns and data validation.
• Knowledge of database testing, including PostgreSQL.
• Experience with load testing and capacity planning for distributed systems.
• Exposure to security testing methodologies and tools.

Skills & Competencies

• Quality mindset; understands that AI systems require different validation approaches than traditional software.
• Strong leadership skills; able to build, motivate, and retain a high-performing QA team.
• Analytical thinker who can design test strategies that maximize coverage with efficient resource use.
• Collaborative; works effectively with engineering, product, and operations teams across geographies.
• Clear communicator; able to articulate quality risks and trade-offs to technical and non-technical stakeholders.
• Self-motivated and effective in a remote environment.
• Fluent in English (written and verbal).

Desired Qualifications

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field.
• ISTQB Advanced or similar QA certification.
• Experience in enterprise software companies or B2B SaaS platforms.
• Background in testing distributed, event-driven, or microservices architectures.

Location & Travel

Location:Hybrid - Selangor or Penang Office

Travel:Minimal; occasional travel for team meetings or training

Language:Fluent English required (written and verbal)

Why Rimini Street?

We are looking for talented, passionate people to help us build our future atRimini Street. We hire only the best, the most extraordinary professionals and provide compensation, bonuses, and benefits to match the skills of our top-performing team members. Do you thrive in a fast-paced environment, enjoy growing together, and get excited about learning new skills? Are you looking for an opportunity to make a true impact as part of a team of extraordinary professionals? This is the place for you.

Our work is challenging and meaningful. We start and end each day with a sense of achievement and purpose guided by our core values, the Four Cs: 

• Company
  • We dream big and innovate boldly.  

• Colleagues
  • We work with extraordinary people who create a culture of mutual respect and collaboration. 

• Clients
  • We relentlessly pursue solutions that help clients achieve their goals. Our unmatched client care is rooted in our passion for exceptional service. 

• Community 
  • We believe in leaving the world a better place than we found it. With the Rimini Street Foundation, we’ve made positive impacts in six continents for over 425 charities.

Accelerating Company Growth

• Nasdaq-listed under ticker symbolRMNI since October 2017 
• Over 6,300+ signed contracts to date, including Fortune 500 and Global 100 companies
• Over 2,000 team members in 23 countries
• US and international recognition for industry leadership and philanthropic efforts. See all of our awards and recognitions here: https://www.riministreet.com/company/awards/ 

Rimini Street is committed to creating a diverse and inclusive environment and is proud to be an Equal Employment Opportunity Employer. All qualified applicants will receive consideration for employment without regard to age, race, color, religion, national origin, sexual orientation, gender or gender identity, disability, protected veteran status, or any other characteristic protected by law. 

To learn more about how Rimini Street is redefining the enterprise software support industry, visithttp://www.riministreet.com 

Please Note: Rimini Street does not accept resumes submitted by recruiting/staffing firms unless specifically requested by Human Resources.  Unsolicited resumes will be ineligible for referral fees.