← All jobs
Expert talent network

AI Evaluation Rubric Auditor

English remote 13 countries — see eligibility below
$60-$85/hour
Verified 2026-11-17
Apply on the official platform

You will continue to the official application page.

The application link may earn OpenJob a commission. It does not affect how listings are ordered or assessed.

About the role

Hourly remote project with the Expert talent network: audit the work of reviewers on AI shopping and delivery tasks and judge whether their rubric failures are fair.

What you will do

Audit complete tasks (prompt plus grading rubric): check whether the information was available, whether the rubric grades what the prompt asked, whether a valid answer would have been marked wrong and whether the rubric follows the guidelines. Then give feedback to reviewers on prompt and rubric issues. The listing mentions about 15 hours per week.

Experience

Substantial experience writing or reviewing rubrics for LLM evaluations (list the projects when you apply), experience giving feedback to writers or reviewers, consistent availability and precise writing that checks the data instead of trusting the reference answer.

Selection process

Apply on the official Expert talent network page. The listing describes the selection steps; no application was submitted by OpenJob.

Similar open jobs

Browse all open remote AI jobs

Eligibility

Residence in Canada, United States, Argentina, Bolivia, Brazil, Chile, Colombia, Ecuador, Guyana, Paraguay, Peru, Uruguay or United Kingdom (official listing data)

Flexible remote tasks

Fluent in English? Outlier's Project Aether pays for evaluating AI images, video and chat replies. No degree or technical background needed, and applying is free.

How Aether works and how to pass the test →
📢 Get new remote AI jobs for your region every day on Telegram →

Get roles like this one on Telegram

This role lists countries in more than one region: pick the channel for where you live. Two messages a day per channel.

🌎 Join OpenJob Americas🌍 Join OpenJob Europe

​