Threat Detection Specialist, Trust and Safety - USDS

This role will sit within the USDS Policy Implementation team and support USDS. You will need to cultivate a deep understanding of the platform and how it can be abused by actors to exploit vulnerable populations while championing freedom of expression for our users and ensuring policies protect users of all experiences and backgrounds.

As a Trend Detection Specialist on the USDS Policy Implementation team, you will proactively identify emerging threats and hidden abuse patterns on our platform for the US market. While the Rapid Response team handles active incidents, your focus is on discovering new vulnerabilities and abuse tactics before they escalate. You will need a deep understanding of platform abuse to protect vulnerable populations, balancing user safety with freedom of expression. Acting as the liaison between Policy and Product, you will partner directly with Engineering to turn your findings into automated, scalable detection systems. Additionally, you will collaborate with the global High-Risk Events and Elections and Trend Detection teams to combat risks specifically targeting the US market.

Responsibilities

• Lead comprehensive risk assessments and spearhead proactive trend detection initiatives for high-risk, USDS-led, or supported events.
• Partner closely with USDS Product teams to co-design and refine automated detection models (classifiers, computer vision, and NLP) that target the U.S. market.
• Build and deploy automated workflows using LLMs to move narrative discovery from manual "spot-checking" to scalable, proactive detection.
• Use your findings from the field to provide high-quality signal and training data to Product, ensuring our automated systems stay ahead of adversarial pivots.
• Support active US escalations by proactively identifying secondary trends, coded language, and evolving narratives that have not yet been caught by current detection systems.
• Generate actionable, data-driven intelligence aligned with quarterly business priorities and cross-functional stakeholder Requests for Information (RFIs).
• Identify emerging slang, memes, and cultural shifts in the U.S. market to predict how high-harm actors might bypass existing safety filters.
• Map new patterns in high-harm violations, focusing on "finding what we don't know" to broaden our safety coverage for the US market.
• Pull and synthesize data that validates the existence of emerging risks, providing the technical evidence needed to trigger new Product features or policy updates.
• Act as a technical subject matter expert for the Policy team, explaining how detection models and tools work and where the gaps lie.

Minimum Qualifications

• 3 years in Trust & Safety, Intelligence, or Content Policy with a focus on high-harm issues (e.g., Violent Extremism, Disinformation, or Youth Safety).
• Technical Acumen: Demonstrated ability to use LLMs or other AI/ML tools to build automated solutions for data classification or trend identification.
• Product Savvy: Experience working with Product/Engineering teams to build, test, or iterate on automated detection systems or safety tools.
• Investigative Mindset: A proven "detective" instinct—you enjoy finding the needle in the haystack and connecting disparate data points into a cohesive narrative.

Preferred Qualifications

• Data Proficiency: Strong skills in SQL and data access tools to independently validate trends and measure the scale of emerging risks.
• Start-up Mentality: Experience forming new functions or working in highly ambiguous, fast-paced environments where you must build your own roadmap.
• Technical Skills: Experience with Python or other scripting languages to further automate data collection and analysis.