Language Model Evaluator - Fully Remote Upto $20 HR Part-Time

### About the job **Mercor** connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include **Benchmark**, **General Catalyst**, **Peter Thiel**, **Adam D'Angelo**, **Larry Summers**, and **Jack Dorsey**. **Position:** Generalist - English & Bengali **Type:****Contract** **Compensation:****$15–$20/hour** **Location:****Remote** ### Role Responsibilities * Conduct fact-checking using trusted public sources and external **tools**. * Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies. * Assess reasoning quality, clarity, tone, and completeness of responses. * Ensure model responses align with expected conversational behavior and system guidelines. * Work **independently and asynchronously** to meet deadlines while improving **AI model performance**. ### Qualifications **Must-Have** * ****Bachelor's degree**** * ****Native speaker** in **Bengali**** * ****Significant experience using large language models** (LLMs)** * ****Excellent writing skills in English**** * ****Strong attention to detail**** * **Background or experience in domains requiring **structured analytical thinking** (e.g., research, policy, analytics, linguistics, engineering)** ### **Preferred** * **Prior experience with **RLHF, model evaluation, or data annotation work**** * **Experience writing or editing **high-quality written content**** * **Experience comparing multiple outputs and making **fine-grained qualitative judgments**** ### **Application Process (Takes 20–30 mins to complete)** * **Upload resume** * **AI interview based on your resume** * **Submit form** ### **Resources & Support** * **For details about the interview process and platform information, please check: https://talent.docs.[mercor](https://himalayas.app/companies/mercor).com/welcome** * **For any help or support, reach out to: support@[mercor](https://himalayas.app/companies/mercor).com** **_PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity._** Originally posted on [Himalayas](https://himalayas.app)