# Machine Learning Engineer - Model Evaluation & Experimentation

> Weekday AI · United States (Remote) · Part-time · Posted 2026-07-30

**Workplace:** remote

**Department:** AI Training

## Description

**This role is for one of our clients**

**Compensation: $60-$90 per hour**

Join a pioneering AI initiative focused on building the next generation of evaluation benchmarks for frontier AI models. We are seeking experienced Machine Learning Engineers and Researchers to bring hands-on expertise in model development, experimentation, and evaluation to create rigorous benchmark tasks for advanced AI systems.

In this role, you will design sophisticated, multi-step machine learning challenges inspired by real-world research workflows. From implementing experimental ideas and running training pipelines to analyzing model behavior and validating results, you will help establish high-quality evaluation benchmarks that reveal the strengths and limitations of frontier AI models.

This is a **fully remote, full-time engagement** requiring approximately **35 hours per week**.

## Requirements

### Key Responsibilities

-   Design realistic machine learning benchmark tasks based on research workflows, including model implementation, experimentation, training, evaluation, and performance analysis.
-   Translate open-ended research concepts into structured, reproducible evaluation tasks with clearly defined success criteria.
-   Implement machine learning solutions using **Python**, execute experiments, and produce reference implementations that demonstrate correct methodology and expected outcomes.
-   Develop benchmark tasks involving reinforcement learning concepts such as reward functions, policy optimization, training dynamics, and model behavior where applicable.
-   Evaluate AI-generated solutions by identifying implementation errors, experimental flaws, incorrect reasoning, and unsupported conclusions.
-   Collaborate with AI researchers and fellow subject matter experts to continuously improve benchmark quality, technical rigor, and evaluation consistency.

### Required Qualifications

-   Master's degree, PhD, or equivalent practical experience in **Machine Learning, Computer Science, Artificial Intelligence, Data Science, or another quantitative STEM discipline**.
-   Minimum **1 year of professional experience** in machine learning research, research engineering, applied AI, or another research-intensive technical role.
-   Strong hands-on experience designing, training, evaluating, and optimizing machine learning models through complete experimental workflows.
-   Practical experience conducting machine learning experiments, including experiment setup, hyperparameter tuning, execution, validation, and analysis.
-   Strong understanding of modern **Large Language Models (LLMs)**, their capabilities, limitations, and evaluation methodologies.
-   Proficiency in **Python** and **Git**, with experience working in both script-based and notebook-based development environments.
-   Familiarity with reinforcement learning concepts—including reward functions, policy optimization, and training behavior—is preferred.
-   Experience with AI evaluation, benchmark development, AI training, or task authoring is highly desirable.
-   Excellent analytical thinking, creativity, attention to detail, and the ability to solve complex, open-ended technical problems independently.
-   Strong written communication skills for documenting experimental methodologies and technical findings.
-   Ability to commit approximately **35 hours per week** on a consistent basis.

### Preferred Qualifications

-   Experience developing or evaluating large language models, foundation models, or generative AI systems.
-   Background in reinforcement learning, deep learning, distributed training, or model optimization.
-   Familiarity with benchmark design, AI safety evaluations, or research-quality experimentation.
-   Experience contributing to research publications, open-source machine learning projects, or advanced AI systems.

### Why Join

-   Help shape how next-generation AI systems are evaluated through rigorous machine learning experimentation.
-   Collaborate with leading AI researchers developing frontier evaluation benchmarks.
-   Apply your expertise to improve AI reasoning, model quality, and experimental reliability.
-   Contribute directly to benchmark development that advances the capabilities of state-of-the-art AI systems.
-   Enjoy the flexibility of a fully remote engagement while working on impactful AI research initiatives.

### Equal Opportunity

We are committed to fostering an inclusive and diverse environment where all qualified applicants receive equal consideration. Reasonable accommodations are available throughout the application and engagement process.

### Contract & Engagement Details

-   Independent contractor engagement.
-   Fully remote with flexible working hours.
-   Expected commitment of approximately **35 hours per week**.
-   Project duration may be extended, shortened, or concluded based on project requirements and individual performance.
-   Work does not require access to confidential or proprietary information from any current or former employer.
-   Payments are issued weekly based on approved work completed.
-   At this time, we are unable to support H1-B or STEM OPT candidates.

## Apply

[Apply at Weekday AI](https://apply.workable.com/weekday-1/j/BF32A485B8/apply)

---
Powered by [Workable](https://www.workable.com)