Clinical Expert AI System Evaluator

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: General Clinician (MD/DO) - HLS Expert Pool
Type: Contract
Compensation: $150/hour
Location: Remote

Role Responsibilities

• Develop grading criteria by breaking clinical questions into discrete checkable criteria for consistent model response grading.
• Evaluate clinical dialogues for accuracy safety completeness and appropriate escalation advice.
• Annotate clinical reasoning by recording case work-throughs including differentials considered and rejected.
• Review outputs to flag hallucinated findings dangerous omissions and clinically unsafe advice.
• Author guidelines to define edge cases and standards of care ensuring annotation consistency.
• Write difficult-case questions to probe current model reasoning limits.

Qualifications

Must-Have

• MD or DO with a completed residency in any specialty
• Active unrestricted medical license in your country of practice
• 3 years post-residency clinical experience
• Comfort writing structured clinical rationale
• Written and spoken English fluency
• Minimum 20 hours per week

Preferred

• Board certification in your specialty
• U.S. licensure and familiarity with U.S. standards of care
• Background in primary care internal medicine emergency medicine or hospitalist roles
• Experience in clinical annotation AI evaluation or medical education
• Grading-criteria design or clinical guideline development experience
• Published research or sustained technical writing (please link a sample)

Application Process (Takes 2030 mins to complete)

• Upload resume
• AI interview based on your resume
• Submit form

Resources & Support

• For details about the interview process and platform information please check:
• For any help or support reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.