DevOps Engineer - AI Model Evaluator

### About the job **Mercor** connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include **Benchmark**, **General Catalyst**, **Peter Thiel**, **Adam D'Angelo**, **Larry Summers**, and **Jack Dorsey**. **Position:** DevOps / SRE / Cloud Engineer (Coding Agent Experience) **Type:****Contract** **Compensation:****$85/hour** **Location:****Remote** ### Role Responsibilities * Use frontier **AI coding agents** to complete and evaluate complex **infrastructure engineering** tasks. * Review model-generated implementations involving **cloud platforms**, **Kubernetes**, **CI/CD systems**, and **infrastructure automation**. * Identify bugs, edge cases, reliability issues, and failure modes in model outputs. * Compare outputs from multiple frontier models to assess strengths and weaknesses. * Apply professional engineering judgment to realistic **infrastructure engineering** scenarios. ### Qualifications **Must-Have** * ****2+ years** of professional **DevOps**, **SRE**, or **Cloud Engineering** experience.** * **Experience with **AWS**, **Azure**, **GCP**, **Kubernetes**, **Terraform**, **CI/CD pipelines**, or observability tooling.** * **Regular use of **AI coding agents** like **Cursor**, **Claude Code**, **Codex**, **Windsurf**, **Gemini CLI**, or similar tools.** * **Ability to evaluate model-generated **infrastructure** and **reliability engineering** solutions.** ### **Preferred** * **Experience supporting **production-scale systems**.** ### **Compensation & Legal** * ****$400 per accepted task**.** * **Compensation is tied to accepted work.** ### **Application Process (Takes 20–30 mins to complete)** * **Upload resume** * **AI interview based on your resume** * **Submit form** ### **Resources & Support** * **For details about the interview process and platform information, please check:** * **For any help or support, reach out to:** **_PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity._** Originally posted on [Himalayas](https://himalayas.app)