textoverlay and overlapping subtitles

[Jérôme Laheurte-3]

It looks like textoverlay can only display one text buffer « at a time »; when several subtitle strings should be displayed it just keeps displaying the first it received, until it’s no longer valid (gstbasetextoverlay.c:2575). I have several videos in which people are talking while a sign is displayed and I often miss half the dialogue because of this. Is there an existing fix/workaround? If not I can try my hand at this. I can see two ways of doing it:

1. Change textoverlay to hold a liked list of currently valid text buffers instead of just the latest one, and compose them before rendering.
2. Create a new « subtitlecompose » element that receives text buffers, does the composition, and outputs text buffers containing all currently valid strings.

The obvious advantage of 2 is that it will work with any combination of text overlay / source elements. May be a bit tricky to use in a playbin though; I guess I could use the text-sink property for that. The problem is that is will rely on textoverlay doing the opposite of what it’s doing right now and discard the existing buffer as soon as it receives a new one. Thoughts?

Best regards
Jérome Laheurte


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-4]

Le dimanche 27 novembre 2016 à 13:42 +0100, Jérôme Laheurte a écrit :
> It looks like textoverlay can only display one text buffer « at a
> time »; when several subtitle strings should be displayed it just
> keeps displaying the first it received, until it’s no longer valid
> (gstbasetextoverlay.c:2575). I have several videos in which people
> are talking while a sign is displayed and I often miss half the
> dialogue because of this. Is there an existing fix/workaround? If not
> I can try my hand at this. I can see two ways of doing it:

Yes, see https://bugzilla.gnome.org/show_bug.cgi?id=540131

>
> 1. Change textoverlay to hold a liked list of currently valid text
> buffers instead of just the latest one, and compose them before
> rendering.
> 2. Create a new « subtitlecompose » element that receives text
> buffers, does the composition, and outputs text buffers containing
> all currently valid strings.
>
> The obvious advantage of 2 is that it will work with any combination
> of text overlay / source elements. May be a bit tricky to use in a
> playbin though; I guess I could use the text-sink property for that.
> The problem is that is will rely on textoverlay doing the opposite of
> what it’s doing right now and discard the existing buffer as soon as
> it receives a new one. Thoughts?

It should manage it's own linked list. The reason is because text-
overlay still need to manage the text placement. If we follow how
assrender works, we'll be stacking up or down (depending on the
location of the overlay) each text. So they form paragraphs. When there
is two strings and one expires, the element need to update the old one,
as it's position may change (or not). This obviously depends on the
rules we want to follow of course, but in general, I believe that if
the screen get saturated, you want to pop out the oldest one. A good
start point would be to look at what assrender is doing.

The idea of having a downstream element composing the overlay is
something I have discussed already with few people. Though, we already
have an element that can handle this, it's called compositor. The
blending operation are the same, it's really a matter of extracting,
converting and caching the overlay buffers. Same apply to glvideomixer.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Jérôme Laheurte-3]

> Le 27 nov. 2016 à 17:45, Nicolas Dufresne <[hidden email]> a écrit :
>
> Le dimanche 27 novembre 2016 à 13:42 +0100, Jérôme Laheurte a écrit :
>> It looks like textoverlay can only display one text buffer « at a
>> time »; when several subtitle strings should be displayed it just
>> keeps displaying the first it received, until it’s no longer valid
>> (gstbasetextoverlay.c:2575). I have several videos in which people
>> are talking while a sign is displayed and I often miss half the
>> dialogue because of this. Is there an existing fix/workaround? If not
>> I can try my hand at this. I can see two ways of doing it:
>
> Yes, see https://bugzilla.gnome.org/show_bug.cgi?id=540131

Thanks; my search-fu seems to be rusty.

>
>>
>> 1. Change textoverlay to hold a liked list of currently valid text
>> buffers instead of just the latest one, and compose them before
>> rendering.
>> 2. Create a new « subtitlecompose » element that receives text
>> buffers, does the composition, and outputs text buffers containing
>> all currently valid strings.
>>
>> The obvious advantage of 2 is that it will work with any combination
>> of text overlay / source elements. May be a bit tricky to use in a
>> playbin though; I guess I could use the text-sink property for that.
>> The problem is that is will rely on textoverlay doing the opposite of
>> what it’s doing right now and discard the existing buffer as soon as
>> it receives a new one. Thoughts?
>
> It should manage it's own linked list. The reason is because text-
> overlay still need to manage the text placement. If we follow how
> assrender works, we'll be stacking up or down (depending on the
> location of the overlay) each text. So they form paragraphs. When there
> is two strings and one expires, the element need to update the old one,
> as it's position may change (or not). This obviously depends on the
> rules we want to follow of course, but in general, I believe that if
> the screen get saturated, you want to pop out the oldest one. A good
> start point would be to look at what assrender is doing.

Okay, I’ll take a look next week-end.

>
> The idea of having a downstream element composing the overlay is
> something I have discussed already with few people. Though, we already
> have an element that can handle this, it's called compositor. The
> blending operation are the same, it's really a matter of extracting,
> converting and caching the overlay buffers. Same apply to glvideomixer.

Maybe a bit much for someone who only occasionally fiddles with GStreamer code :)

Best regards
Jérôme Laheurte


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel