Engineering Analyst, AI Answers, Google Search

About the job

At Google we work hard to earn our users’ trust every day. Gaining and retaining this trust is critically important to Google’s success. We defend Google's integrity by fighting spam, fraud and abuse, and develop and communicate state-of-the-art product policies. The Trust and Safety team reduces risk and protects the experience of our users and business partners across Google's expanding base of products. We work with a variety of teams from engineering to legal and public policy to set policies and combat fraud and abuse in a scalable way, often with an eye to finding industry-wide solutions. Trust and Safety team members are motivated to find innovative solutions, and use technical know-how, user insights and proactive communication to pursue the highest possible quality and safety standards for users across Google products.

Minimum qualifications:

• Bachelor's degree or equivalent practical experience.
  
• 2 years of experience in data analysis, including identifying trends, generating summary statistics, and drawing insights from quantitative and qualitative data.
• 2 years of experience utilizing Python and SQL to solve engineering problems.
• Experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).

Preferred qualifications:

• Experience working with abuse, spam, fraud or malware.
• Experience working on product policy analysis and identifying policy risks.
• Experience working with CBRN and cyber threat risks in GenAI.
• Experience using statistical analysis and hypothesis testing.
• Experience analyzing ML models performance or working on LLMs.
• Strong influencing skills and ability to work in a dynamic environment.

Responsibilities

• Help to establish metrics to benchmark Generative-AI issues and quantify improvements. Develop datasets and run evals for engineers to evaluate and improve Search, Labs & Assistant products.
• Perform complex data analyses to derive insights that help identify and fight abuse across our products. Analyze and debug escalations and identify trends that can indicate bigger product risks. 
• Use technical judgement to drive project delivery, challenge proposals and identify risks. Manage multiple interdependent projects of moderate to high complexity and scope against crashed timelines.
• Improve tools through data analysis, technical expertise, and presentations to key stakeholders. Enhance our operational workflows via process improvements and identification of automation opportunities.
• Review graphic, controversial, and sometimes offensive/sensitive content. Collaborate with other teams & functions: RAI, Policy, Product, etc.