`We did not find an alpha in the model named: F0::Wemb_QuantMultA.` when translating from `pt` to `de` on outbound translations. about firefox-translations

[jelmervdl]

I've had problems with the en-de model as well. Looks like the precomputed alphas may be missing from the model?

[kpu]

I was able to reproduce the error using the files downloaded from here:
https://storage.googleapis.com/bergamot-models-sandbox/0.2.10/ende/model.ende.intgemm.alphas.bin
https://storage.googleapis.com/bergamot-models-sandbox/0.2.10/ende/lex.50.50.ende.s2t.bin
https://storage.googleapis.com/bergamot-models-sandbox/0.2.10/ende/vocab.deen.spm

The command I used to reproduce the error is

~/marian-dev/build/marian-decoder --relative-paths -m moz/model.ende.intgemm.alphas.bin -v moz/vocab.deen.spm{,} --beam-size 1 --mini-batch 32 --maxi-batch 100 --maxi-batch-sort src -w 128 --skip-cost --shortlist moz/lex.50.50.ende.s2t.bin --cpu-threads 1 --gemm-precision int8shiftAlphaAll <<<"Hello"

These files are outdated. They are version 1 of the en-de system not version 2. Also the model file doesn't match version 1 exactly. In any case v2 should be pulled from https://data.statmt.org/bergamot/models/deen_v2.0/ .

I tried to determine the provenance of the model file, but it doesn't match any currently hosted model, v1 or v2.

md5sum v?/*/model.intgemm.alphas.bin moz/model.ende.intgemm.alphas.bin 
c482cd68e65a3cff6b66ac1c79ad4bee  v1/ende.student.base/model.intgemm.alphas.bin
f48744b967863d4e27f68d3e7199d922  v1/ende.student.tiny11/model.intgemm.alphas.bin
f48744b967863d4e27f68d3e7199d922  v1/ende.student.tiny.for.regression.tests/model.intgemm.alphas.bin
977f09cb9781d37c61e9ca45929178c1  v2/ende.student.base/model.intgemm.alphas.bin
f74fc9b331d6fe9f395721a1717a2117  v2/ende.student.tiny11/model.intgemm.alphas.bin
17447b6ef127f5a13fc38415181f115d  moz/model.ende.intgemm.alphas.bin

The vocabulary file is from v1:

md5sum v?/*/vocab.deen.spm moz/vocab.deen.spm
bbbc1f3a2d1dd39e6e88e00f13ef2f23  v1/ende.student.base/vocab.deen.spm
bbbc1f3a2d1dd39e6e88e00f13ef2f23  v1/ende.student.tiny11/vocab.deen.spm
bbbc1f3a2d1dd39e6e88e00f13ef2f23  v1/ende.student.tiny.for.regression.tests/vocab.deen.spm
5dd2fd1c2f5f67e7d84092c5037404f9  v2/ende.student.base/vocab.deen.spm
5dd2fd1c2f5f67e7d84092c5037404f9  v2/ende.student.tiny11/vocab.deen.spm
bbbc1f3a2d1dd39e6e88e00f13ef2f23  moz/vocab.deen.spm

Once I switched the model file to version 2 from https://data.statmt.org/bergamot/models/deen_v2.0/ende.student.tiny11.tar.gz , the command completed successfully

[kpu]

Unassigning self, leaving for @abhi-agg to do a model pull (I don't have permissions to do that).

[kpu]

browsermt/students#46

Sep 14, 2021

"Updated en-de models posted, thanks @kaleidoescape

    "checksum": "7f6bdcf60555fca479e014a6722729b34890e52ca8bfbffb5138f574ec91aec7",
    "url": "http://data.statmt.org/bergamot/models/deen/ende.student.base.tar.gz",

    "checksum": "5214a434a8b6d0562eb927ff5ffe42d4a60240370a0095e0c1369d960878254f",
    "url": "http://data.statmt.org/bergamot/models/deen/ende.student.tiny11.tar.gz",

cc @andrenatal @lonnen"

[kpu]

I note the following agenda items from the Bergamot plenary.
30 September 2021 point 4 "Mozilla pulling in shifted alphas models and config? They’ve been available for a while…"
16 September 2021 point 2.a "en-de updated with WMT21 system"

[jerinphilip]

how can we ensure if the other models we have are current and updated then if no one from your team notifies and replace then in the modelregistry whenever there's a new version then?

We do not have visibility into your model pushing mechanisms. Here's my recommendation. Create a JSON file similar to how https://translatelocally.com/models.json. Use this to generate your modelRegistry. The JSON is already pullable by a python repository mechanism which can be used for continuous testing. Or you can bring your own custom repository in (python) code without having to bundle everything. The merits are manifold:

1. Mozilla's evaluations (https://github.com/mozilla/firefox-translations-evaluation) use python. It'll be easy to pick up for data-viz and table generations over there.
2. All active models can be tested for continuous stability, I think this might even be feasible via GitHub Actions. We can do weekly cron runs against expected output.
3. The model is available to the common public for command line explorations as well. There are models in Mozilla repository that are not available in the browsermt repository.

[abhi-agg]

Update: I applied #61 and tested the workflow. The issue pertaining to not finding the alphas is gone.

Now I see a new error as follows (attached screenshot contains more details):

Translation error:  TypeError: message.sourceParagraph.trim is not a function translationWorker.js:120:37
    consumeTranslationQueue moz-extension://2c65be99-3a64-4361-adf9-ee979fba6c71/controller/translation/translationWorker.js:120
TypeError: message.sourceParagraph.trim is not a function
translationWorker.js:94:72
Translation error:  TypeError: message.sourceParagraph.trim is not a function translationWorker.js:120:37
    consumeTranslationQueue moz-extension://2c65be99-3a64-4361-adf9-ee979fba6c71/controller/translation/translationWorker.js:120

This error seems to be happening only for outbound translation now as the in page translation of PT <-> DE works without any issue now.

[andrenatal]

@abhi-agg I'm working to stabilize and fix all outbound translation issues on this patch: #55. If you apply it you'll see this issue gone.

[abhi-agg]

Awesome. It means I don't need to debug this issue. Once you merge that, we will close this issue.