Bug 161421 - Not all hyphenation separators (hyphens) are displayed in app, but are visible in blue in PDF export / print
Summary: Not all hyphenation separators (hyphens) are displayed in app, but are visibl...
Status: VERIFIED FIXED
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
24.8.0.0 alpha1+
Hardware: x86-64 (AMD64) All
: high normal
Assignee: Heiko Tietze
URL:
Whiteboard: target:25.2.0 target:24.8.0.0.beta2 t...
Keywords: bibisected, bisected, regression
Depends on:
Blocks: Hyphenation
  Show dependency treegraph
 
Reported: 2024-06-04 19:59 UTC by Stefan_Lange_KA@T-Online.de
Modified: 2024-07-14 11:06 UTC (History)
3 users (show)

See Also:
Crash report or crash signature:


Attachments
zip file containing test document, screenshots and exported pdf (2.89 MB, application/x-zip-compressed)
2024-06-04 19:59 UTC, Stefan_Lange_KA@T-Online.de
Details
zip file with screenshots as addition to Comment 5 (1.15 MB, application/x-zip-compressed)
2024-06-23 20:37 UTC, Stefan_Lange_KA@T-Online.de
Details
PDF export with comment 15 build (15.70 KB, application/pdf)
2024-07-09 00:06 UTC, Stéphane Guillou (stragu)
Details
2 Screenshots and 2 pdf documents from print resp. export (4.71 MB, application/x-zip-compressed)
2024-07-09 09:28 UTC, Stefan_Lange_KA@T-Online.de
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan_Lange_KA@T-Online.de 2024-06-04 19:59:01 UTC
Created attachment 194547 [details]
zip file containing test document, screenshots and exported pdf

Some hyphenation separators are not displayed by Writer but instead spaces. As far as I have found until now it relates words hyphenated by Extras - Language - Hyphenation but not words hyphenated by Paragraph setting in Text flow - Hyphenation: Automatic.
The separators are present but "only" hidden (not displayed, spaces instead): When the document is opened in LO 24.2 the separators are displayed and they are also present when the document is exported to pdf by LOdev 24.8 Writer.

reproduced e.g. with
Version: 24.8.0.0.alpha1+ (X86_64) / LibreOffice Community
Build ID: fbe57382eef1138999f63e01b6152d4d05749807
CPU threads: 4; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL threaded

To show the problem I have added a zip file containing a odt test document, screenshots and  and a pdf document exported from the test odt document.
- Test document: Test_Silbentrennung_Trennzeichen_V7a.odt
- compare screenshots Hyph_Seps_LO_24.2.4.jpg and Hyph_Seps_LOdev_24.8.0.jpg to see the differences: many hyphenation separators on line ends with 24.2 vs. nearly no hyphenation separators with 24.8

To show that hypenation separators are hidden by Extras - Language - Hyphenation:
- open test document Test_Silbentrennung_Trennzeichen_V7a.odt
- select the text e.g. below subtitle 1.1.1.2
- open the hyphenation dialog by Extras - Language - Hyphenation and choose "Hyphenate" (in german "Trennen") or "Hyphenate to all" ("Alle trennen")
- Result:  Hyphenation separators displayed before have disappeared

Vice versa one can make appear the not displayed hyphenation separators by activating Paragraph setting "Hyphenation: automatic" and and removing all hyphenations by Extras - Language - Hyphenation and "Remove" ("Entfernen") for every singe place (seems "Remove to all" is not available).
Comment 1 Stefan_Lange_KA@T-Online.de 2024-06-04 22:46:17 UTC
concerning the information "Extras - Language - Hyphenation": In the english version this is "Tools - Language - Hyphenation"

I have tried to bisect the behavior but the Master version in Github is too old (4 weeks).
Comment 2 Stefan_Lange_KA@T-Online.de 2024-06-04 22:48:03 UTC Comment hidden (obsolete)
Comment 3 Stefan_Lange_KA@T-Online.de 2024-06-08 11:37:16 UTC
The Problem first occured in master/Win-x86_64@tb77-TDF in the build from 2024-05-25_03.57.4:
Version: 24.8.0.0.alpha1+ (X86_64) / LibreOffice Community
Build ID: b227481b63000b556e648ad607ffeb86681482a0
CPU threads: 4; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL threaded 

In the build from 2024-05-24_03.59.47 the problem was still not present:
Version: 24.8.0.0.alpha1+ (X86_64) / LibreOffice Community
Build ID: 322b158c5c517cb0242e95240928949ab41d95a0
CPU threads: 4; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL threaded
Comment 4 Dieter 2024-06-23 10:55:35 UTC
Stefan, thank you for reporting the bug, but I can't confirm it with

Version: 24.8.0.0.alpha1+ (X86_64) / LibreOffice Community
Build ID: d2eab48f697a1e6097778158f623f11306ac7a3d
CPU threads: 4; OS: Windows 10 X86_64 (10.0 build 19045); UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: en-GB
Calc: CL threaded

My steps:
1. Open Test_Silbentrennung_Trennzeichen_V7a.odt from attachment 194547 [details]
2. Select some text (e.g. below subtitle 1.1.1.2)
3. Tools -> Language -> Hyphenation -> Hyphenate All

Actual and expected result:
Hyphenation still visible
Comment 5 Stefan_Lange_KA@T-Online.de 2024-06-23 20:35:48 UTC
(In reply to Dieter from comment #4)
> Stefan, thank you for reporting the bug, but I can't confirm it with
> 
> Version: 24.8.0.0.alpha1+ (X86_64) / LibreOffice Community
> Build ID: d2eab48f697a1e6097778158f623f11306ac7a3d
> CPU threads: 4; OS: Windows 10 X86_64 (10.0 build 19045); UI render:
> Skia/Raster; VCL: win
> Locale: de-DE (de_DE); UI: en-GB
> Calc: CL threaded
> 
> My steps:
> 1. Open Test_Silbentrennung_Trennzeichen_V7a.odt from attachment 194547 [details]
> [details]
> 2. Select some text (e.g. below subtitle 1.1.1.2)
> 3. Tools -> Language -> Hyphenation -> Hyphenate All
> 
> Actual and expected result:
> Hyphenation still visible

This is strange because it is a build created after the date I have seen this behavior first.
I have made a new test with the newest bild and the erronous behavior still exists, see screenshots in attached zip file "bug 161421 new screenshots (1).zip".
Version: 24.8.0.0.beta1+ (X86_64) / LibreOffice Community
Build ID: d681c57ba51b635ba7c85b21e062732110e8293f
CPU threads: 4; OS: Windows 11 X86_64 (10.0 build 22635); UI render: Skia/Raster; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: threaded
Newest change is: d681c57 tsan: fix data race in VCLXWindowImpl by Noel Grandin · 32 hours ago libreoffice-24-8 -> Fri Jun 21 14:04:20 2024 +0200

In my screenshots I have coloured the places where hypens are displayed (light green) and where they are "hidden" (dark yellow).

In my new test I have also seen that also hyphens hidden before are are displayed after hyphenate all + undo.
Comment 6 Stefan_Lange_KA@T-Online.de 2024-06-23 20:37:58 UTC
Created attachment 194919 [details]
zip file with screenshots as addition to Comment 5
Comment 7 Stéphane Guillou (stragu) 2024-06-24 13:31:58 UTC
reproduced in:

Version: 24.8.0.0.beta1+ (X86_64) / LibreOffice Community
Build ID: 1b61abc4451d38984338b750d85770ec9871060a
CPU threads: 8; OS: Linux 6.5; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: CL threaded

Using dict-de-de-frami-2017-01-12

Bibisected with linux-64-24.8 repo to first bad build [51e77fbe6053014344651519c29d426f53e08015] which is:

commit 1e9a5ee8671f9a48894849d3cba4d65afa181554
author	Heiko Tietze 	Thu May 23 14:58:46 2024 +0200
committer	Heiko Tietze 	Fri May 24 16:30:31 2024 +0200
Resolves tdf161196 - Visibility of formatting marks
Reviewed-on: https://gerrit.libreoffice.org/c/core/+/167987

Heiko, can you please have a look?
Comment 8 Stéphane Guillou (stragu) 2024-06-24 13:32:25 UTC
(Heiko, forgot to CC you, please see above)
Comment 9 Heiko Tietze 2024-07-05 07:50:53 UTC
I fixed the visibility of non-printable characters for bug 161196. You need to enable it per View > Formatting Marks or ctrl+F10 and fine-tune via Tools > Options > Writer > Formatting Marks. => NAB
Comment 10 Stefan_Lange_KA@T-Online.de 2024-07-06 09:12:58 UTC
(In reply to Heiko Tietze from comment #9)
> I fixed the visibility of non-printable characters for bug 161196. You need
> to enable it per View > Formatting Marks or ctrl+F10 and fine-tune via Tools
> > Options > Writer > Formatting Marks. => NAB

With enabled and fine tuned visibility of non-printable characters it looks good for me and I can accept this solution.
Nevertheless I would think about if it is right if  an "active" soft hyphen (means it is not only a marked place for hyphenation but the word is really hyphenated on this place) is handled as a non-printable character. Because this hyphen is really printed, exported etc. I would see and also handle it as a printable character.
Comment 11 Stéphane Guillou (stragu) 2024-07-08 05:44:32 UTC
(In reply to Heiko Tietze from comment #9)
> I fixed the visibility of non-printable characters for bug 161196. You need
> to enable it per View > Formatting Marks or ctrl+F10 and fine-tune via Tools
> > Options > Writer > Formatting Marks. => NAB
Heiko, did you have a look at the sample document? The hyphens in question _are_ printable characters. They are visible in PDF export and print, but they are not in app or print preview: there's a gap where they should appear. And they appear blue in export.

To be clear:

Testing with attachment 194547 [details], formatting marks off, look at the end of line "fein gerändelte Knöp­fe":

Before 1e9a5ee8671f9a48894849d3cba4d65afa181554:
- printable hyphen visible in app and print preview
- visible in PDF export and print, just as in app

Since 1e9a5ee8671f9a48894849d3cba4d65afa181554:
- printable hyphens not visible in app nor in print preview
- visible in PDF export and print (and in Print dialog's preview), but as blue overlay

I tested with and without dict-de-de-frami installed, and with fresh user profiles.

--- alternatively, to test from scratch:

1. (In default English paragraph, 12 pt, Liberation serif, A4 page) Insert dummy text (dt + F3)
2. Select all
3. Tools > Language > Hyphenation > Hyphenate all

Result: hyphen for "Al-ley" and "fall-ing" not visible on page, but visible in blue in PDF export.
If text is justified, it makes it obvious where the hyphens should appear.
Comment 12 Heiko Tietze 2024-07-08 07:48:30 UTC
Indeed an annoying regression.
Comment 13 Commit Notification 2024-07-08 15:39:27 UTC
Heiko Tietze committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3d0411ef53ccd6bb0af21b69aa557f1f40beffd0

Resolves tdf#161421 - Keep soft hyphen always visible

It will be available in 25.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Commit Notification 2024-07-08 17:57:49 UTC
Heiko Tietze committed a patch related to this issue.
It has been pushed to "libreoffice-24-8":

https://git.libreoffice.org/core/commit/4924908d99194c9541b7791c0e7413a5af332e09

Resolves tdf#161421 - Keep soft hyphen always visible

It will be available in 24.8.0.0.beta2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 15 Stéphane Guillou (stragu) 2024-07-09 00:04:57 UTC
Thanks Heiko, I tested it in my own build. It works as expected on canvas, .uno:PrintPreview, Print dialog preview and Print to File, but I still get the blue hyphens in the PDF export.

Tested with new profile and default PDF export settings.

Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 3d0411ef53ccd6bb0af21b69aa557f1f40beffd0
CPU threads: 8; OS: Linux 6.5; UI render: default; VCL: gtk3
Locale: en-AU (en_AU.UTF-8); UI: en-US
Calc: CL threaded
Comment 16 Stéphane Guillou (stragu) 2024-07-09 00:06:55 UTC
Created attachment 195176 [details]
PDF export with comment 15 build
Comment 17 Stefan_Lange_KA@T-Online.de 2024-07-09 03:18:48 UTC
Tested with
Version: 24.8.0.1.0+ (X86_64) / LibreOffice Community
Build ID: 0cbb17b4cc2af9cdebebb20126a3459be70f09d2
CPU threads: 4; OS: Windows 11 X86_64 (10.0 build 22635); UI render: Skia/Vulkan; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL threaded
Result: not completely OK
- displayed in LO (canvas):  
  all (soft) hyphens are displayed - "visible" as well as "non-visible" (word not hyphenated on the place of hyphen -> hyphen not printed or exported)
    Ctrl+F10 on: blue with gray background
    Ctrl+F10 off: black (resp. character colour) without background
  -> acceptable for me
  -> but better: "non-visible" soft hyphens not displayed at Ctrl+F10 off 
- export to pdf:
  all (soft) hyphens appear blue
  -> not OK
- print (tested with dopdf -> pdf document)
  all (soft) hyphens appear black
  -> OK
Comment 18 Stefan_Lange_KA@T-Online.de 2024-07-09 03:21:59 UTC
Correction to Comment 17:
- print (tested with dopdf -> pdf document)
  all (soft) hyphens appear black - resp. character colour -
  -> OK
Comment 19 Stéphane Guillou (stragu) 2024-07-09 03:43:45 UTC
(In reply to Stefan_Lange_KA@T-Online.de from comment #17)
>   all (soft) hyphens are displayed - "visible" as well as "non-visible"
> (word not hyphenated on the place of hyphen -> hyphen not printed or
> exported)
>     Ctrl+F10 on: blue with gray background
>     Ctrl+F10 off: black (resp. character colour) without background
>   -> acceptable for me
>   -> but better: "non-visible" soft hyphens not displayed at Ctrl+F10 off 
Can you please provide a minimal sample document that has visible + non-visible soft hyphens? I am still a bit new to the topic, and I want to make sure that the issue you described is actually link to Heiko's commit.
Comment 20 Stefan_Lange_KA@T-Online.de 2024-07-09 06:58:13 UTC
(In reply to Stéphane Guillou (stragu) from comment #19)
> (In reply to Stefan_Lange_KA@T-Online.de from comment #17)
> >   all (soft) hyphens are displayed - "visible" as well as "non-visible"
> > (word not hyphenated on the place of hyphen -> hyphen not printed or
> > exported)
> >     Ctrl+F10 on: blue with gray background
> >     Ctrl+F10 off: black (resp. character colour) without background
> >   -> acceptable for me
> >   -> but better: "non-visible" soft hyphens not displayed at Ctrl+F10 off 
> Can you please provide a minimal sample document that has visible +
> non-visible soft hyphens? I am still a bit new to the topic, and I want to
> make sure that the issue you described is actually link to Heiko's commit.
The test document Test_Silbentrennung_Trennzeichen_V7a.odt is such a smalltalk document. I am not abholen currently but when I am back I will send Fürther Information.
Comment 21 Stefan_Lange_KA@T-Online.de 2024-07-09 09:28:28 UTC
Created attachment 195179 [details]
2 Screenshots and 2 pdf documents from print resp. export

Sorry for the crippled text in Comment 20! I should have switch off the input correction of my mobile phone.
 
As written the test document "Test_Silbentrennung_Trennzeichen_V7a.odt" is a small sample document. In all 4 "sections" (camera descriptions) there are active (visble/printable) hyphens as well as inactive (invisible/non printable) hyphens.

Maybe I don't use the exact technical term, therefore:
- As an active (visble/printable) hyphen I see a soft hyphen when the word is hyphenated on his place. It is printed and exported.
- As an inactive (invisble/non printable) hyphen I see a soft hyphen when the word is not hyphenated on his place. It is not printed and also not exported.

In the 2 screenshots one can see the difference in the displayed document with Ctrl+F10 (Formatting Marks) on and off. 

I am not sure if LibreOffice at all can check if a soft hyphen is "active" or "inactive" when it sets the character attributes (character and background colour, visible, ...). I think this can be done only then when setting of chars attributes is made after text is arranged on canvas. Otherwise there is propably no chance to check this.
Comment 23 Commit Notification 2024-07-10 11:47:36 UTC
Heiko Tietze committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/a9e23d1dcd361d95e046ab2d521ee55df9a77747

Resolves tdf#161421 - Draw soft hyphen in black on PDF export

It will be available in 25.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 24 Stefan_Lange_KA@T-Online.de 2024-07-11 08:04:37 UTC
Tested with
Version: 25.2.0.0.alpha0+ (X86_64) / LibreOffice Community
Build ID: 37217909f2e7c042eab9a8b5eb1ab0a88cdda513
CPU threads: 4; OS: Windows 11 X86_64 (10.0 build 22635); UI render: Skia/Vulkan; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL threaded
Result: Hyphens are black now - OK!
Comment 25 Commit Notification 2024-07-13 18:14:21 UTC
Heiko Tietze committed a patch related to this issue.
It has been pushed to "libreoffice-24-8":

https://git.libreoffice.org/core/commit/2968bd73276e9bb62656a5277774926bf7fbd51a

Resolves tdf#161421 - Draw soft hyphen in black on PDF export

It will be available in 24.8.0.2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 26 Stefan_Lange_KA@T-Online.de 2024-07-14 11:06:02 UTC
Tested with
Version: 24.8.0.1.0+ (X86_64) / LibreOffice Community
Build ID: c13bd9d995d82a4f35122fbf1a99516799e83819
CPU threads: 4; OS: Windows 11 X86_64 (10.0 build 22635); UI render: Skia/Vulkan; VCL: win
Locale: de-DE (de_DE); UI: de-DE
Calc: CL threaded

Result: OK!
Hyphens are black now (resp. in the set character colour) - OK also in 24.8!