Job Category: Coding & STEM

We are seeking Life Sciences Subject Matter Experts to design advanced scientific-reasoning tasks for LifeSciBench. This is a non-coding role focused on biological knowledge, experimental interpretation, data analysis, and rigorous scientific reasoning. You will create challenging, objectively defensible problems based on realistic research scenarios, experimental results, biological data, figures, sequences, structures, and literature-style evidence. Main tasks listed: Design novel, graduate-level life sciences problems grounded in realistic research contexts; Develop clear questions and rigorous reference answers with defensible scientific reasoning; Interpret experiments, datasets, figures, sequences, structures, and other biological evidence; Validate biological claims, calculations, assumptions, and conclusions.

We are seeking Earth Sciences experts to develop realistic, terminal-based scientific tasks for Terminal Bench Science. You will create computational workflows involving environmental data, climate systems, atmospheric processes, geophysics, oceanography, geology, and related fields. You will design tasks that require AI agents to inspect scientific datasets, process geospatial or time-series data, run models, troubleshoot pipelines, and generate objectively verifiable scientific outputs. Main tasks listed: Translate authentic Earth-science workflows into self-contained terminal benchmark tasks; Prepare geospatial, climate, atmospheric, geological, hydrological, or oceanographic datasets; Build reproducible computational environments with appropriate scientific libraries and command-line tools; Create expert solutions using Python, R, Bash, Julia, or domain-specific software. The listing says the work can involve sensitive or graphic content.

We are seeking experienced Electrical Engineers to design, solve, and review simulation-led engineering tasks used to train and evaluate advanced AI models. The role requires strong technical fundamentals, practical simulation expertise, and the ability to produce clear, reproducible engineering solutions. We are hiring for a PAA-based project, seeking candidates with strong hands-on experience in CAD modeling and simulation tools. Main tasks listed: Develop CAD models and perform simulation/analysis for aero, mech and elec components; Validate designs using industry-standard tools (e.g., CATIA, SolidWorks, ANSYS, MATLAB); Develop realistic electrical engineering problem statements, simulation models, calculations, schematics, and supporting technical assets; Define inputs, assumptions, constraints, operating conditions, output metrics, and objective acceptance criteria.

We are seeking a highly qualified Art Domain Expert to evaluate and improve Large Language Models (LLMs). You will design challenging prompts, assess factual accuracy and reasoning, and identify knowledge gaps within fine arts, visual arts, architecture, design, art history, museums, and artists. Main tasks listed: Create advanced prompts covering fine arts, visual arts, architecture, design, art history, museums, and artists; Evaluate AI-generated responses for factual accuracy, reasoning quality, completeness, and nuance; Identify hallucinations, logical inconsistencies, outdated information, and edge cases; Develop benchmark datasets and adversarial test cases.

We are seeking a highly qualified TV Shows & Movies Domain Expert to evaluate and improve Large Language Models (LLMs). You will design challenging prompts, assess factual accuracy and reasoning, and identify knowledge gaps within film, television, streaming, actors, directors, genres, awards, franchises, and entertainment history. Main tasks listed: Create advanced prompts covering film, television, streaming, actors, directors, genres, awards, franchises, and entertainment history; Evaluate AI-generated responses for factual accuracy, reasoning quality, completeness, and nuance; Identify hallucinations, logical inconsistencies, outdated information, and edge cases; Develop benchmark datasets and adversarial test cases.

We are building an evaluation benchmark for frontier AI browsing agents. Your job is to design research problems that a state-of-the-art AI cannot solve, even with full web access and multiple attempts. This is not a subject matter expert role, nor is it a content-writing role. It is investigative research. You will start from a verifiable fact, work backwards to construct a question that makes that fact extremely hard to locate, and then prove your work with a complete, auditable evidence trail.

In this role, you will be working on projects to help fine-tune large language models (like ChatGPT) using your strong analytical and English comprehension skills. The ideal candidate should have a solid foundation in STEM, particularly at the level expected in engineering entrance exams, as well as graduate or PhD-level programs. You should be able to break down complex STEM concepts into simple, clear explanations and work efficiently. Main tasks listed: Design and solve challenging STEMs problems to probe the limitations of large language models; Create clear, high-quality, step-by-step solutions with well-articulated reasoning; Collaborate with LLM researchers to align problems with evaluation goals, especially in areas where models typically struggle (e.g., abstraction, multi-step reasoning, symbolic manipulation); Help define new evaluation benchmarks based on Physics curricula spanning early undergraduate to PhD-level topics.

In this role, you will be working on projects to help fine-tune large language models (like ChatGPT) using your strong analytical and English comprehension skills. The ideal candidate should have the ability to read, summarize, and break down large content into smaller logical blocks, conduct research online, validate claims made in content through online research, and work with the LLM (Large Language Models) to solve puzzles! The projects will also help you learn how to leverage AI to be a better analyst. Main tasks listed: You would spend time answering a variety of interesting analytical questions and creating scenarios that can train the LLM models to get better. Here are a couple of examples that models might get wrong, and You would articulate the correct answer & explanation to enable the models to learn:; Based on a given distribution of sales by month across locations, could you analyze which location has grown the most? (Hint: what time period should we look at? should we account for sudden variability at the beginning?); In a small town, there are four distinct neighborhoods: Oak, Pine, Maple, and Elm. A postman is assigned to deliver mail and can only deliver to two neighborhoods in one day, with certain rules (e.g. Oak is always visited before Pine). If he delivers to Oak and Elm on the first day, which neighborhoods does the carrier deliver to on the second day?; Note: No other prior specialized domain experience is needed.

Design original and challenging mathematics problems that test the reasoning limits of large language models, especially in multi-step, abstract, and proof-based settings. Solve problems independently and write detailed, logically structured solutions with clear justifications. Main tasks listed: Design original and challenging mathematics problems that test the reasoning limits of large language models, especially in multi-step, abstract, and proof-based settings; Solve problems independently and write detailed, logically structured solutions with clear justifications; Review model-generated solutions, identify mathematical errors or missing arguments, and provide precise feedback, annotations, and corrections.; Contribute to defining new evaluation benchmarks based on Mathematics curricula spanning from early undergraduate to PhD-level topics.

In this role, you will be working on projects to help fine-tune large language models (like ChatGPT) using your strong analytical and english comprehension skills. The ideal candidate should have a solid foundation in Physics, particularly at the level expected in engineering entrance exams, as well as graduate or PhD-level programs. You should be able to break down complex Physics concepts into simple, clear explanations and work efficiently. Main tasks listed: Design and solve challenging Physics problems to probe the limitations of large language models; Create clear, high-quality, step-by-step solutions with well-articulated reasoning; Collaborate with LLM researchers to align problems with evaluation goals, especially in areas where models typically struggle (e.g., abstraction, multi-step reasoning, symbolic manipulation); Help define new evaluation benchmarks based on Physics curricula spanning early undergraduate to PhD-level topics.