# Medical Expert AI Evaluation Specialist

> Weekday AI · United States (Remote) · Part-time · Posted 2026-07-08

**Workplace:** remote

**Department:** AI Training

## Description

**This role is for one of our clients**

**Compensation: $130-$180 per hour**

### Location requirements

Canada, United States

Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced medical professionals to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.

In this role, you will help create realistic clinical scenarios, evaluate AI-generated responses using evidence-based medical standards, and provide expert insights that improve how AI systems reason about diagnosis, treatment, and patient care.

This opportunity is ideal for physicians across a broad range of medical specialties who are passionate about applying their expertise to the future of healthcare technology.

## Requirements

### Key Responsibilities

### Develop Clinical Scenarios

-   Create realistic, specialty-specific clinical cases that reflect real-world medical practice.
-   Design prompts covering areas such as:

-   Diagnostic reasoning
-   Treatment planning
-   Risk stratification
-   Differential diagnosis
-   Guideline-based clinical decision-making

### Create Reference-Quality Solutions

-   Produce high-quality clinical responses that reflect attending physician-level reasoning.
-   Ensure recommendations align with current evidence-based practices and accepted clinical guidelines.

### Evaluate AI Performance

-   Assess AI-generated medical responses using structured evaluation frameworks and scoring rubrics.
-   Identify strengths, weaknesses, omissions, and reasoning errors.
-   Provide detailed written feedback to improve model performance and clinical accuracy.

### Collaborate with Clinical Experts

-   Participate in onboarding sessions and specialty calibration meetings.
-   Work with multidisciplinary research teams to maintain consistency and quality across medical evaluations.

### Required Qualifications

Applicants should meet one of the following criteria:

### Attending Physicians

-   Board-certified physician with a current, active, and unrestricted medical license.

### Resident Physicians

-   Currently in the final year of an accredited residency program.
-   Recent residency graduates must be board-eligible.

### Fellows

-   Board-certified or board-eligible in their primary specialty.
-   Current, active, and unrestricted medical license.

Physicians from a wide range of specialties are encouraged to apply, including but not limited to:

-   Family Medicine
-   Pediatrics
-   Anesthesiology
-   General Surgery
-   ENT (Otolaryngology)
-   Physical Medicine & Rehabilitation (PM&R)
-   Occupational Medicine
-   Other medical and surgical specialties

### Preferred Qualifications

-   Strong understanding of evidence-based medicine and clinical practice guidelines.
-   Excellent written communication skills.
-   Experience with medical education, clinical research, peer review, or healthcare quality initiatives.
-   Interest in artificial intelligence and healthcare innovation.

### Engagement Details

-   Fully remote with 100% asynchronous work.
-   Typical commitment of approximately 20 hours per week, with opportunities for additional hours based on project needs.
-   Flexible schedule.
-   Weekly payments.
-   Rolling onboarding with ongoing project opportunities.
-   Engagement duration is reviewed periodically based on project requirements and performance.

### Why Join

-   Help shape the future of AI-powered healthcare.
-   Apply your clinical expertise to improve advanced medical reasoning systems.
-   Collaborate with leading AI researchers and healthcare professionals.
-   Contribute to high-impact projects that advance patient-centered artificial intelligence.

### Equal Opportunity Statement

All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.

### Contract Information

-   Independent contractor engagement.
-   Fully remote work completed on your own schedule.
-   Project duration may be extended, modified, or concluded based on business needs and individual performance.
-   Work does not involve access to confidential or proprietary information from any employer, client, or institution.
-   Weekly payments are processed through supported payment platforms based on approved work completed.
-   Please note that visa sponsorship is not available for this opportunity.

## Apply

[Apply at Weekday AI](https://apply.workable.com/weekday-1/j/957FC09B8F/apply)

---
Powered by [Workable](https://www.workable.com)