We are seeking detail-oriented and analytical Business Analysts – Non-US Raters to support a project focused on evaluating and comparing responses generated by multiple AI chatbots. The role involves reviewing chatbot outputs, applying predefined evaluation criteria, conducting structured assessments, and delivering accurate, well-documented judgments.
Candidates should have strong analytical skills, business analysis experience, attention to detail, and the ability to work independently in a remote, fast-paced environment. This opportunity is suitable for professionals interested in contributing to AI evaluation and improving the quality, relevance, and reliability of AI-generated responses.
Job Details
▪Role: Business Analyst – Non-US Raters
▪Experience: 3+ years
▪Employment Type: Short-term contract
▪Contract Duration: 16 weeks
▪Location: Argentina, Peru, Mexico, Brazil, Chile, Colombia, India, and Pakistan
▪Compensation: USD 15 per hour
▪Start Date: Immediate
▪Work Mode: Fully remote
▪Availability: Full-time commitment of 8 hours per day
▪Time Zone Requirement: Minimum 4 hours of overlap with the US Pacific Time (PST) time zone
Show the full posting
▪Work Schedule: Five working days per week, including weekends when required, with two weekdays off
Requirements
Key Responsibilities
▪Review, analyze, and compare responses generated by multiple AI chatbots against predefined evaluation criteria.
▪Conduct structured assessments to determine the accuracy, relevance, completeness, clarity, and overall quality of chatbot responses.
▪Analyze business requirements and translate them into clear evaluation guidelines and functional specifications.
▪Gather, interpret, and document relevant information to support consistent and objective assessments.
▪Examine workflows, processes, and data to identify gaps, inconsistencies, and opportunities for improvement.
▪Prepare evaluation reports, summaries, and supporting documentation with a strong focus on accuracy.
▪Collaborate with project managers and cross-functional teams to clarify requirements and resolve evaluation-related questions.
▪Track task progress, quality metrics, and project deliverables while meeting established deadlines.
▪Identify recurring response-quality issues and provide actionable observations based on the defined assessment framework.
▪Maintain consistency, confidentiality, and high-quality standards throughout the evaluation process.
Required Qualifications and Skills
▪Bachelor's degree in Business Administration, Finance, Computer Science, or a related field.
▪At least 3 years of experience as a Business Analyst or in a similar analytical role.
▪Strong analytical thinking, problem-solving, and decision-making abilities.
▪Excellent written communication skills and the ability to explain findings clearly.
▪Strong attention to detail and the ability to assess information objectively.
▪Proficiency in Microsoft Excel, SQL, Power BI, Tableau, or comparable analytical tools.
▪Experience with requirement gathering, process mapping, and business documentation.
▪Ability to work independently, manage multiple priorities, and meet strict deadlines.
▪Strong documentation skills and the ability to follow structured evaluation guidelines.
▪Familiarity with Agile or Scrum methodologies is an advantage.
▪Willingness to commit to a full-time, 16-week remote engagement with the required time-zone overlap.
Engagement Benefits
▪Opportunity to contribute to projects focused on AI evaluation and response quality.
▪Fully remote work arrangement with the flexibility of working from your location.
▪Exposure to structured AI assessment workflows and quality evaluation processes.
▪Opportunity to contribute to projects with international reach.
▪Clearly defined contract duration, hourly compensation, and evaluation responsibilities.
Important Engagement Details
▪This is a project-based contract lasting 16 weeks.
▪Work will consist of evaluating multiple chatbot responses and submitting a final assessment for each assigned task.
▪Candidates must be available for 8 hours per day, five days per week, with four hours of overlap with US Pacific Time.
▪The schedule may include weekend work, with two weekdays off each week.
▪Candidates are expected to maintain consistent evaluation quality, follow project guidelines, and complete assigned tasks within the required timelines.