Grammarly's mEdIT: fine-tuning multilingual LLMs to support cross-lingual text editing across seven languages
Popular foundational LLMs produce low-quality outputs for text editing tasks, and prior fine-tuning efforts addressed either multiple editing tasks for a single language or a single task across multiple languages—never both simultaneously.
GPT3.5 and GPT4 used as zero-shot baselines performed poorly relative to fine-tuned models on multilingual text editing tasks, with GPT3.5 performing the least well of all models considered.
mEdIT fine-tuned models show a substantial improvement over untrained counterparts across multiple languages and editing tasks, generalize to unseen languages, and received high ratings from human evaluators across all languages.
Show all 6 reported metrics
Frequently asked questions
What did this team achieve with this AI workflow?
mEdIT fine-tuned models show a substantial improvement over untrained counterparts across multiple languages and editing tasks, generalize to unseen languages, and received high ratings from human evaluators across al…
What tools did this team use?
mT5, mT0, BLOOMZ, PolyLM, Bactrian-X, GPT3.5, GPT4, GitHub, Hugging Face.
What results were reported?
Improvement over untrained counterparts: substantial improvement; multilingual vs. English-only instruction training performance: much better; Human evaluator ratings: high ratings across all languages; Performance stability across instruction languages: stable across test cases (source-reported, not independently verified).
What failed first in this deployment?
GPT3.5 and GPT4 used as zero-shot baselines performed poorly relative to fine-tuned models on multilingual text editing tasks, with GPT3.5 performing the least well of all models considered.
How is this quality assurance AI workflow structured?
Curate multilingual training data → Native speaker instruction review → Fine-tune multilingual LLMs → Evaluate against baselines → Collect human evaluator feedback → Publish models and data publicly.