About this job
<h3>About the Role</h3><p style="min-height:1.5em">Join a remote expert network as a senior rater and reviewer, evaluating complex AI model outputs and helping calibrate other graders. Your work supports the development of high-quality, trustworthy AI systems across specialized fields.</p><h3>What You'll Do</h3><ul style="min-height:1.5em"><li><p style="min-height:1.5em">Rank and rate expert-level model answers using detailed rubrics and write clear rationales.</p></li><li><p style="min-height:1.5em">Evaluate steps in model reasoning across fields such as mathematics, coding, law, medicine, and finance.</p></li><li><p style="min-height:1.5em">Fact-check complex claims against primary sources and verify accuracy.</p></li><li><p style="min-height:1.5em">Assess answers against content policies, style guides, and edge cases.</p></li><li><p style="min-height:1.5em">Review other raters' work, calibrate standards, and develop or refine grading guidelines.</p></li><li><p style="min-height:1.5em">Assess search relevance and provide feedback in live sessions.</p></li></ul><h3>What We're Looking For</h3><ul style="min-height:1.5em"><li><p style="min-height:1.5em">At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or a relevant professional domain.</p></li><li><p style="min-height:1.5em">Native or near-native written English proficiency.</p></li><li><p style="min-height:1.5em">Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.</p></li><li><p style="min-height:1.5em">Experience leading or reviewing other raters, such as as a QA lead, rubric owner, or reviewer of reviewers.</p></li><li><p style="min-height:1.5em">An advanced degree, professional license, or deep working expertise in the domain you grade.</p></li><li><p style="min-height:1.5em">Hands-on AI data annotation experience, search-quality rating, or experience writing and maintaining grader guidelines is a plus.</p></li></ul><h3>Compensation & Benefits</h3><p style="min-height:1.5em">Compensation is $50 to $120 per hour, depending on the project, field, and expertise level. Payment is based on hours worked.</p><h3>Location</h3><p style="min-height:1.5em">Fully remote. Projects typically run from one week to several months.</p><br/><br/>Please mention the word **UPLIFTING** and tag RMmEwMTo0Zjg6MTQxOjQyMjk6OjI= when applying to show you read the job post completely (#RMmEwMTo0Zjg6MTQxOjQyMjk6OjI=). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they're human.