Full-time internal role on the platform’s own team, open only to candidates based in India: own cross-functional workstreams that affect the expert experience (onboarding, support, payments, project matching), coordinate Engineering, Operations, Product and Talent Success, turn feedback and support data into prioritized asks, and build lightweight processes and dashboards, using AI tools daily.
This role is part of a project supporting leading LLM companies. The primary objective is to help these foundational LLM companies improve their Large Language Models. We support companies in enhancing their models by offering high-quality proprietary data. This data can be used as a basis for fine-tuning models or as an evaluation set to benchmark the performance. Main tasks listed: Translate CUDA/C++ code into equivalent Python implementations using PyTorch and NumPy, ensuring logical and performance parity; Analyze CUDA kernels and GPU-accelerated code for structure, efficiency, and function before translation; Evaluate LLM-generated translations of CUDA/C++ code to Python, providing technical feedback and corrections; Collaborate with prompt engineers and researchers to develop test prompts that reflect real-world CUDA/PyTorch tasks.
We are seeking an experienced Dermatologist to help evaluate and enhance large language model (LLM) outputs in the medical domain. You’ll assess AI-generated clinical content for dermatological accuracy, safety, and relevance — contributing to the development of responsible AI in healthcare. We are seeking India-based dermatologists to review dermatology images using a structured online evaluation tool. Main tasks listed: Review AI-generated medical text for accuracy, harmfulness, and completeness, with focus on dermatology; Evaluate diagnoses, treatment plans, and summaries for dermatologic cases; Provide expert feedback to improve medical integrity and safety of LLM outputs; Collaborate with AI researchers and clinicians to align model performance with evidence-based dermatology standards. The listing says the work can involve sensitive or graphic content.
We are seeking an AI Prompt & Policy Specialist to help design, evaluate, and improve how AI systems interact with tools, APIs, and users across multiple locales. In this role, you will work at the intersection of prompt engineering, policy design, and safety, ensuring that AI-driven workflows are accurate, compliant, and user-friendly.
We are looking for experienced Data Analysts (MLE Bench) to contribute to benchmark-driven evaluation projects focused on real-world machine learning systems. This role involves hands-on analytical work with production-like datasets, metrics, and ML outputs to help evaluate, diagnose, and improve the performance of advanced AI systems.
We are looking for experienced Software Engineers (SWE Bench – Data Engineer / Data Science) to contribute to benchmark-driven evaluation projects focused on real-world data engineering and data science workflows. This role involves hands-on work with production-like datasets, data pipelines, and data science tasks to help evaluate and improve the performance of advanced AI systems.
We are looking for a Technical Content Writer who can deliver clear, technically accurate content and support data-driven work with hands-on experience using JSON and public datasets. You’ll analyze datasets to extract business insights and present reasoning through well-structured documentation, notebooks, or clearly explained technical artifacts. You will collaborate with researchers and stakeholders, translating complex findings into concise narratives and actionable recommendations.
Job Title: Senior Software Engineer – LLM Evaluation & Repository Validation About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
About the projects: we are building LLM evaluation and training datasets to train LLM to work on realistic software engineering problems. One of our approaches, in this project, is to build verifiable SWE tasks based on public repository histories in a synthetic approach with human-in-the-loop; while expanding the dataset coverage to different types of tasks in terms of programming language, difficulty level, and etc. Main tasks listed: Analyze and triage GitHub issues across trending open-source libraries; Set up and configure code repositories, including Dockerization and environment setup; Evaluating unit test coverage and quality; Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
We’re looking for a ML Developer to drive the design, development, and delivery of advanced machine learning solutions. The ideal candidate is not just a strong individual contributor but also a technical leader capable of setting direction, mentoring team members, and ensuring that ML initiatives align with business goals. Competitive ML experience (e.g., Kaggle, benchmarks) is a strong plus. Main tasks listed: Own end-to-end DS/ML solution development — from data pipelines and model design to deployment and monitoring; Translate business objectives into robust ML architectures that accurately capture business logic and context; Collaborate cross-functionally with Product, Engineering, and Business stakeholders to define problem statements and success metrics; Evaluate and optimize models for performance, scalability, and accuracy using state-of-the-art techniques.