Model Validation Lead - Security & Privacy

About the Team

We are seeking a Model Validation Lead to build the technical capability and team that independently assures the safety and security of all generative AI models used internally and externally by the JV. This is a senior technical individual contributor role, not a people-management position. You will set the technical direction, establish testing methods, lead complex validation engagements, and help develop the specialists and partnerships required for a scalable model-assurance program.

About the Role

You will work closely with TnS to test model safety features and with Penetration Testing and Red Team partners to identify cyber threats created or amplified by generative AI. You will turn emerging attack techniques, model behaviors, and policy requirements into repeatable evaluations that provide evidence of whether safeguards work before deployment and throughout the model lifecycle.

Responsibilities

• Technical Leadership: Serve as the technical lead for model validation without direct people-management responsibility. Define the technical strategy, quality bar, roadmap, and operating model for independent generative AI safety and security assurance.
• Build the Model Validation Capability: Establish the methods, tooling, test infrastructure, and specialist capacity needed to assess every generative AI model used internally or externally by the JV. Help shape hiring profiles, interview technical candidates, mentor practitioners, and coordinate work across a virtual or dedicated team.
• Safety Feature Testing: Partner with TnS to design and execute adversarial evaluations of model safeguards, including harmful-content controls, abuse prevention, age-appropriate protections, policy enforcement, and resistance to jailbreaks or safety-control bypasses.
• Cyber Threat Validation: Work with Penetration Testing and Red Team partners to identify and validate cyber threats stemming from generative AI, including prompt injection, sensitive-data disclosure, insecure tool use, agent manipulation, model and application supply-chain weaknesses, and AI-assisted attacks.
• Independent Model Assessments: Lead pre-deployment and continuous testing of foundation models, fine-tuned models, retrieval-augmented generation systems, agents, and model-enabled products. Assess both intended behavior and foreseeable misuse across internal and external use cases.
• Evaluation Engineering: Build automated, repeatable evaluation pipelines and curated adversarial test suites. Define measurable acceptance criteria, regression thresholds, coverage indicators, and release gates that can be integrated into model development and deployment workflows.
• Threat Modeling and Research: Maintain model-specific threat models and convert emerging research, incidents, and adversary techniques into practical tests. Conduct hands-on experimentation to understand novel failure modes and develop custom tooling where commercial or open-source solutions are insufficient.
• Findings and Remediation: Translate model failures into clear security, safety, privacy, and business risks. Partner with model owners, Engineering, Product, TnS, Privacy, and control owners to recommend mitigations and independently validate remediation.
• Methodology and Governance: Define and maintain Standard Operating Procedures (SOPs), Rules of Engagement (ROE), evidence requirements, and testing methodologies for model safety and security validation. Ensure results are reproducible, auditable, and suitable for risk and release decisions.
• Metrics and Reporting: Produce an authoritative view of model-assurance coverage, safeguard efficacy, unresolved risk, remediation status, and regression trends. Communicate technical findings to engineering teams and convert them into concise decision support for leadership.

Minimum Qualifications

• Bachelor degree in Computer Science, Information Security, Artificial Intelligence, Machine Learning, Computer Engineering, or a related technical field, or equivalent practical experience. 8+ years in security testing, adversarial testing, AI or machine-learning security, trust and safety engineering, red teaming, or a related discipline, including demonstrated experience as a senior technical lead.
• Generative AI Security and Safety: Hands-on experience evaluating large language models or multimodal generative AI systems for adversarial behavior, misuse, safeguard bypasses, data exposure, or unsafe outputs.
• Technical Leadership: Proven ability to set technical direction, lead cross-functional programs, mentor engineers, and build a new capability through influence rather than formal people-management authority.
• Evaluation Design: Experience designing repeatable test plans, adversarial datasets, automated evaluations, and measurable acceptance criteria for complex systems where expected behavior is probabilistic.
• Security Breadth: Strong knowledge of application security, cloud security, offensive security, and threat modeling, with the ability to connect model behavior to risks in the surrounding product, data, identity, and tool-execution layers.
• Coding and Automation: Proficiency in Python and at least one additional language, with experience building test harnesses, integrating model APIs, analyzing results, and automating validation workflows.
• Communication: Ability to explain complex model failures and attack paths to technical and non-technical stakeholders and to provide clear, risk-appropriate remediation guidance.

Preferred Qualifications

• Advanced Model Testing: Experience with jailbreak research, prompt injection testing, agent and tool-use security, retrieval-augmented generation security, multimodal testing, model privacy attacks, or AI supply-chain assessment.
• Trust and Safety Partnership: Experience translating platform or content policies into technical evaluations and working with TnS teams on abuse scenarios, safety taxonomies, and enforcement effectiveness.
• Offensive Security: Experience in penetration testing, red teaming, vulnerability research, exploit development, or adversary emulation, including testing AI-enabled applications in realistic attack chains.
• Frameworks and Standards: Familiarity with the NIST AI Risk Management Framework, MITRE ATLAS, OWASP guidance for large language model applications, and emerging model-evaluation or AI governance standards.
• Research and Community Impact: Contributions to AI safety or security research through publications, open-source tooling, vulnerability disclosures, benchmarks, or presentations at recognized conferences.
• Regulated Environments: Experience conducting independent technical assurance in highly regulated, privacy-sensitive, or national security-focused environments.