Align model outputs with policies via prompt engineering, including mastering TikTok Ad Policies and translating policy knowledge into written definitions, explanations, and prompt instructions.
Create and iterate prompts by translating policy knowledge and labeling standards into structured prompt components.
Perform logical breakdown and evaluation of prompts via root-cause analysis to identify issues and strategize prompt reiteration.
...
Model Quality & Workflow Design: Design, manage, and optimize end-to-end workflows to improve machine model performance in content policy enforcement — including signal detection, rejection accuracy, and leakage reduction. Develop training data pipelines, QA processes, and performance tracking systems aligned to model improvement goals.
Adversarial Testing & Risk Identification: Conduct structured adversarial testing on AI models, features, and content policies to surface vulnerabilities, edge cases, and emerging risk trends. Explore model behaviour across contexts and user journeys to identify failure modes not captured in standard evaluations.
Root Cause Analysis & Error Optimization: Conduct structured root cause analysis (RCA) on model errors — including overkills, leakages, and misclassification — and translate findings into actionable model improvement recommendations. Partner with Algo and product teams to close root causes through memory insertion, threshold adjustments, rewrite rules, or policy iteration.
...
Prompt Engineering: Design and deploy advanced prompting strategies, including Chain-of-Thought (CoT), Meta-Prompting, and Least-to-Most Prompting, to precisely guide model behavior.
AI Tool Integration: Maintain and leverage automated Root Cause Analysis (Auto-RCA) tools and automated Prompt Agents to quickly identify the causes of model failures and generate potential solutions within minutes.
Logic Diagnosis: Conduct deep-dive analysis into model hallucinations and classification errors. You will identify whether failures are caused by tokenization issues, logical gaps, or variance driven by temperature settings.
...