Response Comparison Rater

The Response Comparison Rater role is a part-time, fully remote position focused on the quality and logic of written text. You will look at various outputs from language models and determine which ones communicate most effectively based on a specific set of guidelines. This work helps ensure that the tools being built are helpful, accurate, and free of the vague language that often clutters professional communication.

About the Role In this role, you will help us evaluate the performance of AI models by comparing their generated responses. Olive Tree Consulting values clarity and brevity, and this position supports that goal by identifying which machine-generated answers are truly useful and which are simply filler. You will be looking at prompts and assessing pairs of responses for factual accuracy, logical consistency, and tone. This is an entry-level contract position that requires no previous experience in consulting or data science, but it does require a high level of comfort with the English language and a critical eye for detail. Since this work is conducted entirely online, you can complete your tasks from any location with a reliable internet connection.

What You Will Do

• Review pairs of AI-generated responses to see which one better follows the user instructions.
• Rank responses based on a provided rubric that covers dimensions like truthfulness and clarity.
• Write short, clear justifications explaining why one specific response is superior to another.
• Identify and flag instances where an AI model provides incorrect information or makes up facts.
• Check for "hallucinations" or logical leaps that would make a response untrustworthy.
• Evaluate the formatting and structure of written outputs to ensure they follow professional standards.
• Provide feedback on whether the responses use plain English or rely on unnecessary jargon.
• Consistently meet quality targets by maintaining a high degree of focus across all rating tasks.

What We Are Looking For We need people who can read a text and quickly identify where it goes wrong. You should be able to spot slight contradictions or subtle shifts in tone that might escape an average reader. This role is a good fit for someone who enjoys writing and editing and who can describe their reasoning clearly. You do not need a specific degree, but you should have a strong grasp of grammar and a logical approach to problem-solving. Because this is a remote, part-time position, reliability and the ability to manage your own schedule are essential traits. We value honest assessments over quick ones, so patience is a requirement.

Helpful Background

• Previous experience in writing, editing, or proofreading (this can be academic or personal projects).
• Comfort using basic online productivity tools and navigating simple data entry interfaces.
• An interest in the way language works and how machines attempt to replicate human speech.
• The ability to follow complex, multi-step instructions without getting overwhelmed.
• A background in a field that requires close reading, such as law, history, philosophy, or English.
• Experience working in a remote layout where clear written communication is the primary way to stay in touch.

What We Offer This is a flexible, remote position that allows for a steady flow of work. You will be working at your own pace within agreed timelines, making it a suitable option for those balancing other commitments. The position is paid, and the pay is competitive for the role and market. You will gain direct experience with the processes used to train modern language models and contribute to work that prioritizes clarity over corporate buzzwords. While the work is independent, we provide clear documentation and support to help you understand the rating guidelines.

About Olive Tree Consulting Olive Tree Consulting Group is a focused advisory practice that helps leadership teams solve recurring management issues. We typically work on the space between strategy and daily execution. A team can have very capable people and still find that decisions keep reopening, priorities stall, or responsibility remains unclear. These are rarely personality problems: they are structural habits that have not kept up with the company's growth.

We favor practical changes over large-scale transformation programs. Our work involves looking at how decisions are made, how meetings are run, and how ownership is recorded. We prefer useful written notes and clear ownership over polished presentations and vague alignment. Good advisory work should make things plainer and leave the team with a rhythm they can continue on their own.

The firm is built on the principle that memory is not a management system. We help teams create the smallest possible written record to support their work so that important follow-up does not depend on who was in the room or who has the best memory. The goal is a quieter management pattern that makes the existing work easier to lead.