Java QA Automation Engineer - Banking

Responsibilities:

 Build API and integration test coverage for Java/Spring routers and database workflows.
• Design and implement Playwright E2E tests for critical user journeys.
• Create stable mocking and fixture strategies (MSW, test fixtures, seeded test data).
• Set up CI quality gates for linting, unit/integration/E2E tests, coverage, and reporting.
• Expand coverage to untested UI and workflow areas and prevent regressions.
• Build an LLM evaluation discipline with golden datasets and measurable quality gates.
• Track model output reliability, detect drift, and gate releases on quality/stability thresholds.
• Improve test maintainability, remove flakiness, and optimize runtime.
• LLM QA Tooling
• Use LangChain4j for JVM-native LLM application support, including agents, RAG, and guardrails.
• Use Langfuse Java SDK and OpenTelemetry for trace capture, prompt/version tracking, and experiment comparison.
• Use the OpenAI Java SDK for structured outputs, function calling, webhook verification, and response validation.
• Use Promptfoo in CI when you need language-agnostic regression and red-team checks around the Java stack.
• Use Java schema validation and guardrail patterns with Jackson, Swagger annotations, and LangChain4j guardrails.
• Use OpenTelemetry-based traces and Langfuse datasets as the reproducible source of truth for release gates.

Qualifications:

3+ years in QA automation for production systems.
• Strong Java backend testing skills.
• Strong frontend testing.
• Experience building and maintaining Playwright/Cypress/Selenium E2E suites.
• API testing experience (REST, schema validation, auth/error scenarios).
• Experience with SQL/PostgreSQL testing and test data management.
• CI/CD pipeline experience (GitHub Actions, Azure Pipelines, or Jenkins).
• Practical understanding of testing non-deterministic LLM outputs. Nice to have: Familiarity with Java-compatible LLM evaluation and regression tooling such as Promptfoo and Langfuse, plus tracebased observability and output validation via API/OpenTelemetry integrations.
• Performance and benchmark testing experience.
• WebSocket testing experience.
• Security and dependency scanning awareness.
• Experience in legal-tech, fintech, or other regulated domains.

*We are an equal opportunity employer and value diversity. All employment decisions are made without regard to age, gender, disability, race, ethnicity, religion, sexual orientation, or any other protected characteristic. We encourage applicants from all backgrounds to apply.*