English AI Quality Analyst - Personalization

Help improve personalized AI experiences by testing how well AI uses relevant information from prior conversations and connected online activity. You will design realistic prompts based on your own experiences, compare model responses, and explain which response is more grounded, natural, and helpful.

What You’ll Do

• Design and run short, multi-turn prompts that require meaningful personalization.

• Compare responses side by side and assess grounding, integration, helpfulness, naturalness, and unsupported inferences.

• Write clear, evidence-based rationales that reference specific turns.

• Review debug information and maintain clean evaluation conversations.

Requirements

• Strong English reading and writing ability.

• Willingness to use your primary personal Google account

• Excellent analytical judgment, attention to detail, and concise written communication.

• Ability to work independently on a remote team using a reliable computer and internet connection.

• A bachelor’s degree or equivalent practical experience in an analytical field.

Preferred Background

• At least one year of experience in data annotation, AI quality evaluation, content moderation, prompt evaluation, or related analytical work.

• Experience evaluating nuanced AI responses or writing structured comparative rationales.

Engagement Details

• Remote freelance project for candidates based in the United States.

• Expected duration of approximately four months.

• Choose a 30- or 40-hour weekly commitment, with at least four hours per day and four hours of overlap with Pacific Time.

Compensation

$19 USD per hour.