Expertise: AI Evaluation

We are seeking highly motivated Team Management to support the evaluation and improvement of cutting-edge Large Language Models (LLMs). In this role, you will analyze structured data, create challenging prompts, evaluate AI-generated responses for factual accuracy and reasoning quality, and provide evidence-backed feedback to improve model performance. We are looking for curious, detail-oriented professionals who enjoy solving complex problems and evaluating AI systems. Main tasks listed: Create challenging prompts that evaluate an LLM’s ability to retrieve, analyze, and reason over structured data; Assess AI-generated responses for factual accuracy, logical reasoning, and completeness; Identify model failures, inconsistencies, hallucinations, and reasoning gaps; Validate model outputs using provided datasets and supporting evidence.

We are looking for a detail-oriented QA Analyst to evaluate and compare responses generated by multiple chatbots. The role involves reviewing outputs against defined guidelines, identifying inaccuracies or quality issues, and providing a clear final assessment for each evaluation task. Main tasks listed: Evaluate and compare responses generated by multiple chatbots; Assess responses for accuracy, relevance, completeness, clarity, and adherence to guidelines; Identify errors, inconsistencies, missing information, and potential quality issues; Provide structured feedback and final assessments for each task.

The Text Situated Alignment Generalist – BA role supports a short-term human-AI interaction evaluation pilot designed to assess whether providing social context before a user’s request helps a language model generate more relevant, personalized, and user-aligned advice. Contributors create realistic, ongoing social scenarios, interact with the model under different conditions, and evaluate the quality of its responses. Main tasks listed: Create realistic and ongoing social scenarios involving personal, professional, or interpersonal context; Interact with the language model under assigned evaluation conditions; Evaluate model responses for relevance, usefulness, personalization, clarity, and alignment with the provided context; Explain the reasoning behind each rating through clear and detailed written feedback.

The Text Situated Alignment Generalist – BA role supports a short-term human-AI interaction evaluation pilot designed to assess whether providing social context before a user’s request helps a language model generate more relevant, personalized, and user-aligned advice. Contributors create realistic, ongoing social scenarios, interact with the model under different conditions, and evaluate the quality of its responses. Main tasks listed: Create realistic and ongoing social scenarios involving personal, professional, or interpersonal context; Interact with the language model under assigned evaluation conditions; Evaluate model responses for relevance, usefulness, personalization, clarity, and alignment with the provided context; Explain the reasoning behind each rating through clear and detailed written feedback.

We are looking for participants based in South Korea to support a project involving Google Wallet and Plaid. Main tasks listed: Follow the project instructions and complete assigned tasks using Google Wallet and Plaid; Provide accurate information about the account setup and task experience; Report any issues encountered during the process; Maintain confidentiality regarding all project-related information.

We are looking for participants based in Japan to support a project involving Google Wallet and Plaid. Main tasks listed: Follow the project instructions and complete assigned tasks using Google Wallet and Plaid; Provide accurate information about the account setup and task experience; Report any issues encountered during the process; Maintain confidentiality regarding all project-related information.

We are developing a computer usage dataset to train intelligent systems that understand how users interact with software applications. The project involves recording your screen while performing detailed, multi-step tasks — such as configuring applications, exploring tools, or completing productivity workflows. Each task may include 100+ individual interactions (clicks, keypresses, and other on-screen events). Main tasks listed: Record your screen activity while completing assigned computer-based tasks; Carefully follow provided instructions and documentation to complete each task step-by-step; Ensure recordings are complete, high-quality, and interruption-free; Label or verify events (clicks, keystrokes, actions) when required. This work may involve recording or sharing personal data (for example voice, video or account activity): read the consent and privacy terms on the official page before applying.

As an AI Quality Analyst, you will evaluate a new personalization feature for Gemini. You will assess how well the model uses information from your past Gemini conversations, Gmail, Google Search, and YouTube activity to make responses more relevant and helpful. This role requires a unique blend of creativity and analytical rigor. You will actively design prompts from the perspective of your own personal experiences. This work may involve recording or sharing personal data (for example voice, video or account activity): read the consent and privacy terms on the official page before applying.

Use your research, reasoning, and analytical skills to help improve leading AI models. We’re looking for students and working professionals to research information, solve analytical problems, and evaluate AI-generated responses. 20 hours/week commitment, with eligibility for up to 40 hours/week Main tasks listed: Research online sources and verify information; Analyze complex content and logical problems; Summarize information into clear, concise insights; Review AI-generated responses and identify or correct errors.

As an AI Quality Analyst, you will evaluate a new personalization feature for Gemini. You will assess how well the model uses information from your past Gemini conversations, Gmail, Google Search, and YouTube activity to make responses more relevant and helpful. This role requires a unique blend of creativity and analytical rigor. You will actively design prompts from the perspective of your own personal experiences. This work may involve recording or sharing personal data (for example voice, video or account activity): read the consent and privacy terms on the official page before applying.