Company: Turing

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

Design original and challenging mathematics problems that test the reasoning limits of large language models, especially in multi-step, abstract, and proof-based settings. Solve problems independently and write detailed, logically structured solutions with clear justifications. Main tasks listed: Design original and challenging mathematics problems that test the reasoning limits of large language models, especially in multi-step, abstract, and proof-based settings; Solve problems independently and write detailed, logically structured solutions with clear justifications; Review model-generated solutions, identify mathematical errors or missing arguments, and provide precise feedback, annotations, and corrections.; Contribute to defining new evaluation benchmarks based on Mathematics curricula spanning from early undergraduate to PhD-level topics.

In this role, you will be working on projects to help fine-tune large language models (like ChatGPT) using your strong analytical and english comprehension skills. The ideal candidate should have a solid foundation in Physics, particularly at the level expected in engineering entrance exams, as well as graduate or PhD-level programs. You should be able to break down complex Physics concepts into simple, clear explanations and work efficiently. Main tasks listed: Design and solve challenging Physics problems to probe the limitations of large language models; Create clear, high-quality, step-by-step solutions with well-articulated reasoning; Collaborate with LLM researchers to align problems with evaluation goals, especially in areas where models typically struggle (e.g., abstraction, multi-step reasoning, symbolic manipulation); Help define new evaluation benchmarks based on Physics curricula spanning early undergraduate to PhD-level topics.

In this role, you will be working on projects to help fine-tune large language models (like ChatGPT) using your strong analytical and English comprehension skills The ideal candidate should have a strong foundation in Chemistry, comparable to the level required for PhD-level programs. You should be able to break down complex chemistry problems into simple, clear explanations and work efficiently. The projects will also help you learn how to leverage AI to be a better analyst. Main tasks listed: You would spend time creating and solving a variety of advanced chemistry problems, including those at the postgraduate level, and creating detailed explanations. You’ll regularly work with a mix of chemistry content—combining text with images, chemical equations, diagrams, and visual representations. Here are a few examples of the kinds of tasks you might handle day to day:; Analyze and explain the steps in a complex organic reaction mechanism, providing both textual explanations and annotated chemical structures; Solve a problem involving chemical equilibrium, accompanied by visual representations of reaction progress and energy diagrams, and explain the process using both text and visual aids; Address challenging problems involving stoichiometry, thermodynamics, organic chemistry, and electrochemistry, typical of Engineering Entrance Exam or Engineering College level questions, utilising a combination of textual, visual, and diagrammatic explanations.

In this role, you will be working on projects to help fine-tune large language models (like ChatGPT) using your strong analytical and english comprehension skills The ideal candidate should have a solid foundation in Biology, particularly at the level expected in engineering entrance exams, as well as graduate or PhD-level programs. You should be able to break down complex biological concepts into simple, clear explanations and work efficiently. Main tasks listed: Design and solve challenging Biology problems to probe the limitations of large language models; Develop high-quality, step-by-step solutions with clear and rigorous reasoning; Collaborate with LLM researchers to align problem types with evaluation goals, especially in areas where models typically struggle (e.g., conceptual abstraction, multi-step reasoning, and data interpretation); Contribute to defining new evaluation benchmarks based on Biology curricula spanning early undergraduate to PhD-level topics.

As a Software Engineering evaluator, you will create cutting-edge datasets for training, benchmarking, and advancing large language models, collaborating closely with researchers. This includes curating code examples, providing precise solutions, and making corrections in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go; evaluating and refining AI-generated code for efficiency, scalability, and reliability; and working with cross-functional teams to enhance enterprise-level AI-driven coding…

About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.