Research

Which models translate Korean webtoon dialogue best, and at what cost?

Measured
2026-11-18
Sample
38 lines, 10 models, 760 scores
Data
2 CSV files, CC BY 4.0

In our November 2026 blind test on 38 Korean webtoon lines, Claude Opus 5.5 scored 8.77 of 10 and DeepSeek V4.1 Flash 1017 scored 8.68 at one 36th of the cost. Seven of the ten models scored between 7.59 and 8.25. Every model scored lower on sound effects than on dialogue. At this size, differences under about 0.3 points are noise.

Table of contents
  1. What did we test?
  2. How did we test it?
  3. Which models scored best?
  4. What do the results show?
  5. Download the data
  6. Licence and how to cite

What did we test?

Translation and the edit pass after it are the two most expensive model calls in a Bublate chapter. We tested whether cheaper models can do them as well as the model in production, Claude Opus 5.5, with everything else held the same.

How did we test it?

  • Lines. Two sets from episode 0 of the Korean webtoon Mercenary Enrollment: a sample set of 15 lines and a held-out set of 23 lines. 28 lines are dialogue (speech, shouts and thoughts) and 10 are sound effects. We publish no text from the chapter.
  • Pipeline. Bublate's production code for translate and polish, with the settings of the quality profile. Only the model changed, with no fallback model. All models got one shared scene brief.
  • Polish. Each model polished its own drafts. Models that read images polished with the pages in view. Three text-only models polished without them.
  • Judging. Two blind judges per set scored every line from 0 to 3 against a key of meaning and style marks, paraphrased from the official English release. The judges saw the models under shuffled letters.
  • Score. The mean line score, divided by 3 and multiplied by 10. Each model has 76 scores: 38 lines, 2 judges.
  • Cost. Each model's real tokens on OpenRouter for both sets, thinking included.

Which models scored best?

ModelScore of 10DialogueSound effectsScores at 3Scores at 0 or 1Test costCheaper than Opus
Claude Opus 5.5 (production)8.778.998.1767%3$0.29191x
DeepSeek V4.1 Flash 10178.689.117.5068%6$0.008136x
Meta Muse Ember8.258.637.1754%5$0.020614x
GPT-6 Luna8.168.517.1749%3$0.007042x
Tencent Hy4 Flash8.168.577.0053%6$0.008037x
Gemma 4 31B7.948.107.5049%8$0.003877x
Solar Pro 4 (text only)7.898.336.6747%8$0.008933x
MiniMax M3.1 Flash7.727.747.6747%11$0.008833x
Hy-MT2 30B (text only)7.598.046.3338%8$0.0026113x
Mi:dm K 3.0 Pro (text only)6.806.856.6726%13$0.014720x

Mi:dm K 3.0 Pro left 2 lines empty. Tencent Hy4 Flash wrote 49,294 output tokens and took 325.5 seconds for both sets, the slowest in the test.

What do the results show?

  • One cheap model matched the production model. DeepSeek V4.1 Flash 1017 came within 0.09 points of Claude Opus 5.5 and scored higher on dialogue. Its weak lines were of the same kind: an idiom, a sound effect choice.
  • Sound effects are the hard part. Every model scored lower on the 10 sound effect lines than on dialogue. The best sound effect score was 8.17.
  • Price is not quality. The cheapest model, Hy-MT2 30B, scored 7.59. Meta Muse Ember cost more than every model except Claude Opus 5.5 and scored third.
  • Low scores tell the story. Claude Opus 5.5 and GPT-6 Luna had 3 scores of 0 or 1 out of 76. Mi:dm K 3.0 Pro had 13.

Bublate still translates with Claude Opus 5.5. A model change needs a full chapter and more than one run per model first. For how the series bible keeps names and voice the same across chapters, read inside the series bible.

Download the data

Each file is a CSV with a header row, comma separated, UTF-8. Measured: 2026-11-18. The files hold numbers and descriptions only, no art or text from the test chapters.

Score and cost per model

One row per model: overall, dialogue and sound effect scores, score counts, test cost, tokens and time.

translation-models-summary-2026-11.csv · 10 rows · CSV

Columns
ColumnMeaning
modelModel id on OpenRouter
score_of_10, dialogue_score_of_10, sfx_score_of_10Mean line score divided by 3 and multiplied by 10: all lines, dialogue lines, sound effect lines
judgementsScores the model got (38 lines, 2 judges)
pct_judgements_at_3, judgements_at_0_or_1Share of scores at 3, and count of scores at 0 or 1
test_cost_usd, times_cheaper_than_opusModel cost of both sets on OpenRouter, and Claude Opus 5.5 cost divided by it
tokens_in, tokens_out, secondsTokens sent and received, thinking included, and time for both sets
polish_saw_pagesyes when the edit pass saw the page images
lines_left_emptyLines the model returned no text for

Every score

One row per line, model and judge: 38 lines, 10 models, 2 judges.

translation-models-scores-2026-11.csv · 760 rows · CSV

Columns
ColumnMeaning
setsample (15 lines) or heldout (23 lines)
lineLine id: segment and block number in the chapter
line_kindspeech, shout, thought or sfx
modelModel id on OpenRouter
judgejudge_1 or judge_2 of that set
score_0_to_3The judge's score for the line

Licence and how to cite

The data is free to reuse under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence. You can copy, change and publish it, also for commercial use, if you credit Bublate and link to this page. Suggested citation:

Bublate team (2026). Korean webtoon translation model test (November 2026). Bublate. https://bublate.com/research/translation-model-test. Data licensed CC BY 4.0.

First published . If we correct or extend the data, the files and this page change together and the updated date above moves.

Frequently asked questions

Which model translates Korean webtoons best?

In our test of 38 lines, Claude Opus 5.5 scored highest at 8.77 of 10, and DeepSeek V4.1 Flash 1017 scored 8.68. A gap under about 0.3 points is noise at this size, so the two are level on this test. One chapter and one run cannot settle it for every series.

Is a cheaper model good enough?

One was on this test. DeepSeek V4.1 Flash 1017 scored within 0.09 points of Claude Opus 5.5 at about one 36th of the cost. The other cheap models scored 6.80 to 8.25. Test on a full chapter of your own series before you switch.

Why are sound effects scored lower?

A sound effect has no single right word. The translator must pick the English sound that fits the scene, the size of the art and the house style. Every model in the test scored lower on the 10 sound effect lines than on its dialogue lines.

Who were the judges?

Two blind judges per set scored each line from 0 to 3 against a key of meaning and style marks, paraphrased from the official English release. They saw the candidate lines under shuffled letters, so they could not tell which model wrote which line.

Can I reuse this data?

Yes. The CSV files are under the Creative Commons Attribution 4.0 licence. You can copy, change and publish them, also for commercial use, if you credit Bublate and link to this page.

Sources

  1. OpenRouter, models and prices · checked
  2. Creative Commons, Attribution 4.0 International · checked

Try Bublate for free

Continue with Google

or