Conversational Agent Audio Evaluation

Overview

I need help evaluating conversational agent audio responses by comparing them against a reference voice. The focus is on audio delivery, not content meaning.

Scope of work

- Listen to a short conversation context (audio + transcript).
- Compare two agent responses (audio samples).
- Evaluate each response across specific dimensions: faithfulness, expressiveness, speech controllability, voice consistency, overall quality.
- Select which response is overall better or mark them tied.
- Use the discard option if technical issues occur or if responses are in an unfamiliar language or read punctuation aloud.

Additional information

Focus only on audio quality and delivery, not the content meaning. Responses may vary between single-turn or multi-turn conversations. Numbers are yet to be finalized for each location or language.

USER MUST BE FAMILIAR WITH TESTING LANGUAGE (PREFERABLY IT AS MOTHER TOUNGUE)

User testing language

French, Italian, German, Spanish, Hindi, Japanese, Korean, Portuguese, Chinese (simplified)