LLM QA Reviewer / AI Output Validator (RAG Systems)

Description

We are building a small network of specialists focused on AI reliability and LLM validation.

At ProoflineAI, we work with companies deploying AI assistants and RAG-based knowledge systems. Our goal is to ensure these systems are accurate, grounded, and production-ready.

This role is focused on evaluating AI outputs—not building models.

What You’ll Do

You will help assess and improve AI system reliability by:

• Reviewing LLM responses for factual accuracy
• Detecting hallucinations and fabricated references
• Verifying grounding against source documents
• Identifying retrieval vs generation failures in RAG systems
• Scoring responses using structured evaluation criteria
• Creating test prompts and edge cases
• Documenting failure patterns clearly

Ideal Profile

You may be a strong fit if you have experience in:

• AI QA / AI annotation
• NLP or LLM evaluation
• QA testing / data quality
• ML Ops / AI operations
• Reviewing AI-generated content critically

Strong written English and attention to detail are essential.

Location (Preferred)

We are primarily looking to work with candidates based in:

Poland
Romania
Portugal
Estonia
Latvia
Lithuania

Compensation

Depending on workload:

• €800 – €1,200/month (part-time)
• €1,200 – €1,800/month (full-time)

Long-term collaboration possible based on performance.