-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
span elements are included in NBG(R_i) #5
Comments
@palemieux please could you explain more about how this would work? I don't understand the reasoning at the moment: The paint text model is based on glyph copying, and the glyphs are copied after the background has been drawn. The background is not the same for every glyph with the listed style properties, even if Even if the background were the same size, the rendering performance factor for drawing a background would need to be taken into account, and presumably would be much less than for drawing the glyph: in this proposal what change would be made to the values of |
@nigelmegitt Do you agree that the current complexity measure, where a full region redraw is assumed on every span-specified |
@palemieux it depends what you need to achieve. I do agree that real implementations are unlikely to redraw all of the pixels in a region n times where n is the number of background colours specified for active children of that region. At least, it's very much a worst case scenario. I could imagine some actually do have event loops that might cause this many redraws, if they update some background painting map every time they encounter a new background rectangle. But as a proxy measure of complexity, which is easy to compute, I think it is okay to assume this worst case scenario, generate some number (which is all the HRM does, after all), and then set thresholds based on the numbers that real world content requires. Sure, we could go to a huge amount of further detail, computing the areas of each of the elements selected into the region, and multiplying the number of pixels by the number of times each element gets drawn etc. Let me ask you a more meta question back: what is the motivation behind increasing the level of precision? Is there some real world content that colours a lot of span backgrounds differently and falls foul of the HRM? |
The Timed Text Working Group just discussed
The full IRC log of that discussion<nigel> Topic: span elements are included in NBG(R_i) w3c/imsc#571<nigel> github: https://github.com/w3c/imsc/issues/571 <nigel> Nigel: This is about where you count the background colour of spans. <nigel> Pierre: The problem with the current spec is that the cost of drawing a background on span <nigel> .. is the same as the cost of drawing a background on the entire region. <nigel> .. That seems to scale the wrong way. <nigel> .. Intuitively you'd think that the cost of drawing the background of a span <nigel> .. should roughly scale with the number of characters in the span, not the area of the region. <nigel> Nigel: My feeling is "kinda" and "maybe" and "it's not that bad is it?" <nigel> .. I mean, some spans might be as big as a region, but it's unusual. <nigel> .. As a worst case scenario, it's not so bad. We're just looking for a complexity value. <nigel> Pierre: It is bad though because a p with one span, compared to the same p with the same text in multiple spans, <nigel> .. generates a higher complexity value. <nigel> Nigel: You mean there are documents that fail compared to the threshold now, that should pass? <nigel> Pierre: Yes, there are cases where there are 2 documents with the same rendered output, <nigel> .. and one fails and one passes, based on how many spans are in the p output. <nigel> Nigel: It'd be good to see test cases. <nigel> .. Although the rendered output is the same, that doesn't mean that the rendering complexity was the same. <nigel> Pierre: Imagine an implementation that uses fixed width bitmap characters. <nigel> .. The cost of applying a background to a span should never scale as the area of the region, <nigel> .. because the background would scale with every character drawn. <nigel> .. You would not redraw the entire background of the region every time you blit a character. <nigel> .. it would just be the area of the character. <nigel> Nigel: It would be useful to have test cases. Am I right that there are no HRM tests in imsc-tests? <nigel> Pierre: Right, so the project I'm doing could probably add them. <nigel> Nigel: Any other thoughts on this? <nigel> SUMMARY: Continue the discussion on the issue |
@nigelmegitt Below is the kind of document I have in mind:
The The region is transparent and large so that the document can use a single region with no risk of overflow. |
Thanks for that example @palemieux . It clarifies things. I'm still thinking about it, and want to share what I'm mulling over:
|
@nigelmegitt I agree it is too early to reach a conclusion on this issue, and recommend waiting for more feedback from implementers and users. |
I'm at a loss to understand why every span background would cause a cost of a complete region fill... I would expect a new buffer per time period, with a single clear and then draw on. Note on region size: I have generally adopted the use of regions which extend from the top or bottom safe area edge to the required text position. In practice, as in pierre's example, this means large regions for every title. Small region sizes implies you know the text extents, and in this world of 'make my text bigger' in renderers, you can't know the final text sizes as the player may change that. So the best you can do is 'anchor' the text to the position you want it, and encourage the player to expand from there? |
You could alternatively set |
The HRM is not intended to be a representation of a performance-optimised real world renderer - in fact it probably makes sense to make it in some ways a deliberately sub-optimal model, so that even naive implementations are likely to play back conformant documents acceptably. |
As it stands, the complexity of drawing the background of a span remains comparable to drawing the entire background of a parent region that fills the root container, even if the font size of the span is 0. I do not think believe this is correct. |
Yes, this reflects an extremely naive renderer, for example one that proceeds linearly through rendering, writing directly to a single pixel buffer, and redraws all the background areas every time it encounters something that modifies how those background areas should be drawn. I'm not saying that the pointer is necessarily at the right place on the dial now; just that moving it all the way to "highly optimised" would also be a mistake. |
painting the whole region for every background colour makes no sense after reading 10.2, which explicitly groups the painting of all background colours before the painting of foregrounds. |
The ratio of the cost of drawing a background behind a span over the cost of drawing a background behind the root container will approach 0 as the font size approaches 0, even for an extremely naive renderer. |
I don't think that's necessarily correct. The current terrible case naive renderer model builds a map of all the backgrounds, as it progresses linearly through the content, and every time it finds a new one, redraws the whole region background. Anyhow, I think it's a strong point by @btsimonh that the processing requirements for a renderer mean that text layout has to happen in full before background painting can begin. Perhaps it would make sense to define the creation of a set of rectangles in different colours (and transparencies) and count the painting time of each of those, dependent on its visible area. But doing it as part of glyph painting seems problematic, for the reasons stated earlier. Then we'd have:
|
"Perhaps it would make sense to define the creation of a set of rectangles in different colours (and transparencies)" - this is almost exactly what I do in my renderer which I wrote some time back. |
+1 - I expect we'd factor this into the painting "speed" |
Add editorial note re: outstanding issue #5 Change document license
Discussed in TTWG meeting today, minuted at https://www.w3.org/2023/12/21-tt-minutes.html#r01 :
The proposal is to close this issue with no change on the basis that in practice, the potential overestimate of background painting time in the HRM discussed in this issue has not caused any documents to fail validation unexpectedly. If, in the future, implementation and usage experience shows that there are documents that are expected to pass HRM validation but actually fail, then we would analyse those cases and either open new issues or, if the cause is the same, we could reopen this issue. I will return to this after January 8th 2024 and if there are no objections, I will close this issue. |
NBG(Ri) counts the number of
tts:backgroundColor
attributes specifiedspan
elements.In a common scenario illustrated below, this results in the complexity of painting (relatively small) span backgrounds to be equal to painting the background of (relatively much larger) region that essentially fills the root container.
This can be addressed by excluding
span
from the NBG(Ri) computation, and instead includingtts:backgroundColor
in the list of glyph properties at https://www.w3.org/TR/ttml-imsc1.1/#paint-text.The text was updated successfully, but these errors were encountered: