Job Category: Coding & Engineering

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

As a Software Engineering evaluator, you will create cutting-edge datasets for training, benchmarking, and advancing large language models, collaborating closely with researchers. This includes curating code examples, providing precise solutions, and making corrections in Python, JavaScript (including ReactJS), C/C++, Java, Rust, and Go; evaluating and refining AI-generated code for efficiency, scalability, and reliability; and working with cross-functional teams to enhance enterprise-level AI-driven coding…

About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.

Remote contract work for competitive programmers on an AI training project: solve self-contained coding, logic and debugging challenges designed to test advanced AI models, often in unfamiliar codebases or terminal environments, and produce reproducible solutions.

Remote contract work for machine learning engineers on an AI training project: create, solve and validate ML engineering tasks such as implementing or modifying models, building reproducible training or inference workflows, optimizing memory or throughput and debugging numerical or system failures. The listing describes the role as very coding-heavy.

Remote, flexible hourly contract for senior software engineers working on agentic AI: design and refine workflows where AI plans, writes, tests and iterates on code, evaluate AI-generated code for correctness, efficiency and security, create real-world coding challenges and benchmarks, identify failure modes of autonomous agents and improve evaluation rubrics and test harnesses. Task-based, asynchronous work on your own schedule.

Remote hourly contract for senior software engineers: design coding benchmarks and data pipelines used to evaluate frontier AI models, analyse AI-generated code for correctness and quality, build realistic evaluation scenarios and give detailed technical feedback. The listing states a 3-month initial contract with possible extension; full-time availability is preferred.