TranslateGemma Quality Evaluation / Stress Test feat Alex Murauski

Episode 144 • March 03, 2026 • 01:00:52
TranslateGemma Quality Evaluation / Stress Test feat Alex Murauski
Nimdzi LIVE!
TranslateGemma Quality Evaluation / Stress Test feat Alex Murauski

Mar 03 2026 | 01:00:52

/

Show Notes

In this session, we will explore how we evaluated the translation quality of Google’s Gemma model using the MQM framework and a human-in-the-loop review process.

The case study walks through how LLM-generated translations were assessed using structured error typology, how linguistic quality was benchmarked, and how AI-enhanced workflows can combine automated generation with professional post-editing and evaluation.

We’ll discuss:

How MQM works in real-world AI evaluation

What kinds of errors LLMs produce across languages

Where AI performs well — and where it still struggles

How to design scalable human-in-the-loop evaluation workflows

What this means for localization vendors and enterprise buyers

The session is based on a real case study conducted by Alconost’s MT evaluation team using our MQM evaluation tool.

Full case:
https://alconost.mt/mqm-tool/case-studies/translategemma/

Other Episodes

Episode 89

August 24, 2023 • 00:56:16
Episode Cover

Navigating Game Culturalization feat. Kate Edwards

With products, entertainment, and services reaching more customers and audiences around the world each year, lives are becoming just a little more similar. People...

Listen

Episode 112

April 02, 2024 • 01:00:13
Episode Cover

AI for translation & interpretation: What makes sense and what doesn’t

In this episode, we dive once more into the world of AI and its impact on the translation and interpretation industry. Our expert guests,...

Listen

Episode 104

January 15, 2024 • 01:03:41
Episode Cover

Human Meets AI: A chat about "Human Approved" feat. Marta Castello

Join us for a revolutionary episode of The Nimdzi Live Show. We're now diving into uncharted territory with Innovation Manager, Marta Castello from Creative...

Listen