Country Eligibility: United States

We are looking for a detail-oriented Business Analyst to support a project involving the evaluation of multiple chatbot responses. The role will require analyzing business requirements, comparing chatbot outputs, conducting structured evaluations, and providing final assessments based on defined criteria. Main tasks listed: Gather, analyze, and document business requirements; Collaborate with stakeholders to understand business needs and objectives; Translate business requirements into clear functional specifications; Analyze workflows, processes, and data to identify gaps and improvement areas.

Evaluate LLM models for areas of finance where models do not perform well. Create rubrics to assess model capabilities on specific areas of your finance expertise (such as deal analysis, M&A assessments, and more). Main tasks listed: Evaluate LLM models for areas of finance where models do not perform well; Create rubrics to assess model capabilities on specific areas of your finance expertise (such as deal analysis, M&A assessments, and more); Collaborate with AI researchers and fellow finance experts to shape training methods, evaluation strategies, and benchmarks.

We are seeking experienced US-based Medical Doctors (MD) or international equivalent (MBBS, MBChB, Medical Diploma, etc.) to help evaluate and enhance large language model (LLM) outputs in the medical domain. You’ll assess AI-generated clinical content for medical accuracy, safety, and relevance — contributing to the development of responsible AI in healthcare. Main tasks listed: Review AI-generated medical text for accuracy, harmfulness, and completeness, with focus on medical field; Evaluate diagnoses, treatment plans, and summaries for medical cases; Provide expert feedback to improve medical integrity and safety of LLM outputs; Collaborate with AI researchers and clinicians to align model performance with evidence-based medical standards.

We are looking for a detail-oriented QA Analyst to evaluate and compare responses generated by multiple chatbots. The role involves reviewing outputs against defined guidelines, identifying inaccuracies or quality issues, and providing a clear final assessment for each evaluation task. Main tasks listed: Evaluate and compare responses generated by multiple chatbots; Assess responses for accuracy, relevance, completeness, clarity, and adherence to guidelines; Identify errors, inconsistencies, missing information, and potential quality issues; Provide structured feedback and final assessments for each task.

We are seeking Subject Matter Experts (SMEs) with expertise in psychology, social work, counseling, qualitative research, HCI, communication, conflict resolution, coaching, or related advisory fields to support AI conversation evaluation research. SMEs will review conversations between participants and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality. Main tasks listed: Review AI conversations involving real-world interpersonal and everyday decision-making scenarios; Identify missing context and recommend relevant follow-up questions; Evaluate whether additional information would have changed the AI’s recommendations; Distinguish between critical (“load-bearing”) and non-essential information.

We are looking for experienced Software Engineers to help evaluate, benchmark, and improve the coding capabilities of frontier AI models. In this role, you will assess AI-generated code, validate solutions against real-world software engineering tasks, identify correctness and quality issues, and contribute to the development of high-quality evaluation datasets and benchmarks.

We are seeking a highly qualified TV Shows & Movies Domain Expert to evaluate and improve Large Language Models (LLMs). You will design challenging prompts, assess factual accuracy and reasoning, and identify knowledge gaps within film, television, streaming, actors, directors, genres, awards, franchises, and entertainment history. Main tasks listed: Create advanced prompts covering film, television, streaming, actors, directors, genres, awards, franchises, and entertainment history; Evaluate AI-generated responses for factual accuracy, reasoning quality, completeness, and nuance; Identify hallucinations, logical inconsistencies, outdated information, and edge cases; Develop benchmark datasets and adversarial test cases.

Use your research, reasoning, and analytical skills to help improve leading AI models. We’re looking for students and working professionals to research information, solve analytical problems, and evaluate AI-generated responses. 20 hours/week commitment, with eligibility for up to 40 hours/week Main tasks listed: Research online sources and verify information; Analyze complex content and logical problems; Summarize information into clear, concise insights; Review AI-generated responses and identify or correct errors.

Compensation: $100-$150/hour, please provide a specific hourly rate expectation Availability: 20-40 hours/week, with at least 6 hours of Pacific Time overlap/day Main tasks listed: Design, refine, and iterate on rubrics and evaluation criteria for preference data; Review and label data with a high quality bar, catching subtle issues others might overlook; Build and maintain pipelines and infrastructure supporting data generation, collection, and evaluation workflows; Synthesize findings from data work into clear write-ups, updates, and recommendations for the team.

We are seeking an experienced Dermatologist to help evaluate and enhance large language model (LLM) outputs in the medical domain. You’ll assess AI-generated clinical content for dermatological accuracy, safety, and relevance — contributing to the development of responsible AI in healthcare. We are seeking US-based dermatologists to review dermatology images using a structured online evaluation tool. Main tasks listed: Review AI-generated medical text for accuracy, harmfulness, and completeness, with focus on dermatology; Evaluate diagnoses, treatment plans, and summaries for dermatologic cases; Provide expert feedback to improve medical integrity and safety of LLM outputs; Collaborate with AI researchers and clinicians to align model performance with evidence-based dermatology standards. The listing says the work can involve sensitive or graphic content.