Machine translation fails for paragraphs with reference with images
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	santhosh
	Jan 5 2022, 7:18 AM

Description

List of steps to reproduce (step by step, including full links if applicable):

Try to translate https://en.wikipedia.org/wiki/James_Webb_Space_Telescope to another language, say ig using Google translate. Click on the first paragraph.

What happens?:

Google MT fails, provides this stack trace in network debugger

Error: Invalid sourceHtml
    at Google.buildSourceDoc (/srv/service/lib/mt/MTClient.js:457:10)
    at Google.translateReducedHtml (/srv/service/lib/mt/MTClient.js:58:8)
    at Google.translate (/srv/service/lib/mt/Google.js:30:15)
    at MWImage.adapt (/srv/service/lib/translationunits/MWImage.js:78:64)
    at /srv/service/lib/lineardoc/TextBlock.js:458:32
    at Array.forEach (<anonymous>)
    at /srv/service/lib/lineardoc/TextBlock.js:446:9
    at Array.forEach (<anonymous>)
    at TextBlock.adapt (/srv/service/lib/lineardoc/TextBlock.js:443:19)
    at Doc.adapt (/srv/service/lib/lineardoc/Doc.js:774:23)
    at Adapter.adapt (/srv/service/lib/Adapter.js:25:27)
    at MWReference.adapt (/srv/service/lib/translationunits/MWReference.js:80:47)
    at Doc.adapt (/srv/service/lib/lineardoc/Doc.js:735:37)
    at /srv/service/lib/lineardoc/TextBlock.js:466:41
    at Array.forEach (<anonymous>)
    at TextBlock.adapt (/srv/service/lib/lineardoc/TextBlock.js:443:19)

What should have happened instead?:

Machine translation should succeed

Details

	Subject	Repo	Branch	Lines +/-
	Update cxserver to 2022-02-01-141918-production	operations/deployment-charts	master	+1 -1
	Support images generated by templates	mediawiki/services/cxserver	master	+79 -0

Customize query in gerrit

Related Objects

Mentioned In: T292412: Technical exploration for the integration of Flores MT models
T292415: Create Wikipedia Paiwan
T298679: Wrong section segmentation causes infobox presented as table
T298752: cxserver gives empty result for translation of template without about attribute
T298584: Deploy Flores MT in Production
Mentioned Here: T301952: For references with multiple templates, the calculation of overall adaptation status is wrong
T225716: Reference marked as missing but successfully adapted
T292412: Technical exploration for the integration of Flores MT models
T292415: Create Wikipedia Paiwan
T298584: Deploy Flores MT in Production
T298679: Wrong section segmentation causes infobox presented as table
T298752: cxserver gives empty result for translation of template without about attribute

Event Timeline

santhosh created this task.Jan 5 2022, 7:18 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 5 2022, 7:18 AM

A pattern observed for this error is, the paragraph has reference which is a template and that it contains an image.

Change 751676 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Support images generated by templates

https://gerrit.wikimedia.org/r/751676

gerritbot added a project: Patch-For-Review.Jan 5 2022, 7:29 AM

santhosh claimed this task.Jan 5 2022, 7:29 AM

Setting high priority since https://en.wikipedia.org/wiki/Template:Source-attribution or https://en.wikipedia.org/wiki/Template:PD-notice is used very frequently in references for English wiki. If any paragraph has such references, MT is failing for all MT engines.

santhosh added a project: Language-Team (Language-2021-October-December).Jan 5 2022, 9:41 AM

santhosh moved this task from Quarter Backlog to In Review on the Language-Team (Language-2021-October-December) board.Jan 5 2022, 9:43 AM

Pginer-WMF edited projects, added Language-Team (Language-2022-January-March); removed Language-Team (Language-2021-October-December).Jan 10 2022, 3:22 PM

Pginer-WMF moved this task from Quarter Backlog to In Review on the Language-Team (Language-2022-January-March) board.

Change 751676 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Support images generated by templates

https://gerrit.wikimedia.org/r/751676

Maintenance_bot removed a project: Patch-For-Review.Jan 21 2022, 6:10 PM

Change 758862 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2022-02-01-141918-production

https://gerrit.wikimedia.org/r/758862

gerritbot added a project: Patch-For-Review.Feb 1 2022, 2:34 PM

Change 758862 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2022-02-01-141918-production

https://gerrit.wikimedia.org/r/758862

Maintenance_bot removed a project: Patch-For-Review.Feb 1 2022, 3:10 PM

Mentioned in SAL (#wikimedia-operations) [2022-02-01T15:13:34Z] <kart_> Deployed Flores MT for cxserver + Updated cxserver to 2022-01-13-174407-production (T298584, T292412, T292415, T298679, T298752) + Updated cxserver to 2022-02-01-141918-production (T298592)

Stashbot mentioned this in T298679: Wrong section segmentation causes infobox presented as table.Feb 1 2022, 3:13 PM

Stashbot mentioned this in T292415: Create Wikipedia Paiwan.

Stashbot mentioned this in T292412: Technical exploration for the integration of Flores MT models.

• Nikerabbit moved this task from In Review to Needs QA on the Language-Team (Language-2022-January-March) board.Feb 3 2022, 8:05 AM

There is no error in the console, but the references are not properly adapted. They are shown as empty references (rendered in grey) when they are not empty. This may cause publishing to skip them. The expected result would be for these references to be rendered in blue instead as regular references.

ig.wikipedia.org_wiki_Special_ContentTranslation_from=en&to=ig&page=James+Webb+Space+Telescope&targettitle=Telescopio+espacial+James+Webb(iPad Air).png (1×2 px, 693 KB)

This may be a regression of T225716, or a case not covered by it.

Pginer-WMF moved this task from Needs QA to In Progress on the Language-Team (Language-2022-January-March) board.Feb 11 2022, 8:42 AM

In T298592#7703201, @Pginer-WMF wrote:

There is no error in the console, but the references are not properly adapted. They are shown as empty references (rendered in grey) when they are not empty. This may cause publishing to skip them. The expected result would be for these references to be rendered in blue instead as regular references.

I checked this and this is an unrelated issue: T301952: For references with multiple templates, the calculation of overall adaptation status is wrong

Great! Thanks @santhosh

	F34948200: ig.wikipedia.org_wiki_Special_ContentTranslation_from=en&to=ig&page=James+Webb+Space+Telescope&targettitle=Telescopio+espacial+James+Webb(iPad Air) copy.png
	Feb 11 2022, 8:41 AM

	F34948198: ig.wikipedia.org_wiki_Special_ContentTranslation_from=en&to=ig&page=James+Webb+Space+Telescope&targettitle=Telescopio+espacial+James+Webb(iPad Air).png
	Feb 11 2022, 8:41 AM

	F34907991: image.png
	Jan 5 2022, 7:20 AM

Machine translation fails for paragraphs with reference with imagesClosed, ResolvedPublicBUG REPORTActions

Description

Details

Related Objects

Event Timeline

Machine translation fails for paragraphs with reference with images
Closed, ResolvedPublicBUG REPORT
Actions