gstreamer and video device probing

[Filippo]

Now that HAL is going away I'm looking for a good replacement for the
v4l device detection code in Cheese.
At the moment we ask HAL to give us a list of devices from the
video4linux subsystem and we do a couple of ioctls to see if the
device is a v4l2 or v4l1 one and if it supports the CAPTURE
capability. Those ioctls aren't really needed because HAL already does
that stuff for us but I found it out only yesterday :P
Anyway, the purpose of those ioctls is to exclude each non-webcam
device (tuners, radios, vbis, etc) before to start the gstreamer
pipeline.
This way we can reduce a lot the time needed for device probing
(starting a v4l*src pipeline can be quite slow) and we can avoid
"garbage" in the device list.
At first I thought I could just replace the HAL stuff with DeviceKit,
but DeviceKit is going away too in favor of plain libudev [1].

So the next step should be to port Cheese to libudev, but as David
suggested [2], why not to do the detection directly in Gstreamer?
As far as I can tell the current option with Gstreamer is to use the
PropertyProbe interface.
But as it is at the moment it's suboptimal for our needs:
- the device detection code in v4l?src just tries to open
/dev/v4l/video* and /dev/video* devices. It doesn't even check the V4L
api they support.
Wouldn't be better to use libudev to enumerate all the devices from
the video4linux subsystem?

- there seems to be no way to get video device capabilities without
starting a pipeline, both v4lsrc and v4l2src implement (and not in a
good way) propertyprobe just for the "device" property. Would it be
possible to use propertyprobe for "flags" property too? is it possible
to fix a property (i.e. the device) and probe another one (i.e. the
flags)?
What I'd like to do is:
pspec = gst_property_probe_get_property (probe, "device");
GValueArray *array = gst_property_probe_probe_and_get_values (probe, pspec);
for (i = 0; i < array->n_values; i++) {
     g_object_set_property (G_OBJECT (element), "device", device);
     pspec = gst_property_probe_get_property (probe, "flags");
     /* get flags for the current device */
     ...
}
Linking and playing a simple pipeline for each device just for exclude
radios and tuners is not exactly optimal from my point of view.

- there is no way to monitor new devices attached. At the moment we
don't use such a thing but wouldn't be bad to have it. But does it fit
in gstreamer purpose?

A good solution would be a custom class for device detection, a bit
like the one David suggested [2], that supports several backends (a
libudev one for v4l devices, some win32 thing for win32 sources,
freebsd, opensolaris, etc...).
It could monitor new devices attached, retrieve supported capabilities
(or maybe just filter them by capability directly in the monitor code)
and maybe provide the best suited source element to access it.

What do you think about it? Are there other users for this kind of
stuff other than Cheese? Is Gstreamer the place for this probing?
Is there anyone willing to actually implement it or help to implement it?
Any suggestion or opinion welcome.

Cheers,
Filippo


1. http://lists.freedesktop.org/archives/devkit-devel/2009-April/000140.html
2. http://marc.info/?l=linux-hotplug&m=124181585125194&w=2

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Stefan Sauer]

Filippo Argiolas schrieb:

> Now that HAL is going away I'm looking for a good replacement for the
> v4l device detection code in Cheese.
> At the moment we ask HAL to give us a list of devices from the
> video4linux subsystem and we do a couple of ioctls to see if the
> device is a v4l2 or v4l1 one and if it supports the CAPTURE
> capability. Those ioctls aren't really needed because HAL already does
> that stuff for us but I found it out only yesterday :P
> Anyway, the purpose of those ioctls is to exclude each non-webcam
> device (tuners, radios, vbis, etc) before to start the gstreamer
> pipeline.
> This way we can reduce a lot the time needed for device probing
> (starting a v4l*src pipeline can be quite slow) and we can avoid
> "garbage" in the device list.
> At first I thought I could just replace the HAL stuff with DeviceKit,
> but DeviceKit is going away too in favor of plain libudev [1].
>  

Your patch needs to be applied by someone who can vaerify it. I have no
libudev in my distro :/

> So the next step should be to port Cheese to libudev, but as David
> suggested [2], why not to do the detection directly in Gstreamer?
> As far as I can tell the current option with Gstreamer is to use the
> PropertyProbe interface.
> But as it is at the moment it's suboptimal for our needs:
> - the device detection code in v4l?src just tries to open
> /dev/v4l/video* and /dev/video* devices. It doesn't even check the V4L
> api they support.
> Wouldn't be better to use libudev to enumerate all the devices from
> the video4linux subsystem?
>  

I commented in the bug, maybe we should use this for all kind of
hardware sources and sinks.

> - there seems to be no way to get video device capabilities without
> starting a pipeline, both v4lsrc and v4l2src implement (and not in a
> good way) propertyprobe just for the "device" property. Would it be
> possible to use propertyprobe for "flags" property too? is it possible
> to fix a property (i.e. the device) and probe another one (i.e. the
> flags)?
> What I'd like to do is:
> pspec = gst_property_probe_get_property (probe, "device");
> GValueArray *array = gst_property_probe_probe_and_get_values (probe, pspec);
> for (i = 0; i < array->n_values; i++) {
>      g_object_set_property (G_OBJECT (element), "device", device);
>      pspec = gst_property_probe_get_property (probe, "flags");
>      /* get flags for the current device */
>      ...
> }
> Linking and playing a simple pipeline for each device just for exclude
> radios and tuners is not exactly optimal from my point of view.
>  

I don't think it has to be played. You probably not even need to link it
as READY is enough for getting it to open the device.

> - there is no way to monitor new devices attached. At the moment we
> don't use such a thing but wouldn't be bad to have it. But does it fit
> in gstreamer purpose?
>  
I don't think so, I use HAL in buzztard to detect devices. There is
nothing in gstreamer that is guaranteed to always run.

I hope other developers comment here too.

Stefan

> A good solution would be a custom class for device detection, a bit
> like the one David suggested [2], that supports several backends (a
> libudev one for v4l devices, some win32 thing for win32 sources,
> freebsd, opensolaris, etc...).
> It could monitor new devices attached, retrieve supported capabilities
> (or maybe just filter them by capability directly in the monitor code)
> and maybe provide the best suited source element to access it.
>
> What do you think about it? Are there other users for this kind of
> stuff other than Cheese? Is Gstreamer the place for this probing?
> Is there anyone willing to actually implement it or help to implement it?
> Any suggestion or opinion welcome.
>
> Cheers,
> Filippo
>
>
> 1. http://lists.freedesktop.org/archives/devkit-devel/2009-April/000140.html
> 2. http://marc.info/?l=linux-hotplug&m=124181585125194&w=2
>
> ------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>  

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, &
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[David Zeuthen]

Hi,

On Mon, 2009-06-01 at 14:05 +0300, Stefan Kost wrote:

> Filippo Argiolas schrieb:
> > Now that HAL is going away I'm looking for a good replacement for the
> > v4l device detection code in Cheese.
> > At the moment we ask HAL to give us a list of devices from the
> > video4linux subsystem and we do a couple of ioctls to see if the
> > device is a v4l2 or v4l1 one and if it supports the CAPTURE
> > capability. Those ioctls aren't really needed because HAL already does
> > that stuff for us but I found it out only yesterday :P
> > Anyway, the purpose of those ioctls is to exclude each non-webcam
> > device (tuners, radios, vbis, etc) before to start the gstreamer
> > pipeline.
> > This way we can reduce a lot the time needed for device probing
> > (starting a v4l*src pipeline can be quite slow) and we can avoid
> > "garbage" in the device list.
> > At first I thought I could just replace the HAL stuff with DeviceKit,
> > but DeviceKit is going away too in favor of plain libudev [1].
> >  
> Your patch needs to be applied by someone who can vaerify it. I have no
> libudev in my distro :/

For the record, libudev has been around since 8/27 2008 and it has been
LGPLv2.1 since 3/26 2009. FWIW, today I just added GObject bindings
(including GObject introspection support and a Javascript example)

 http://people.freedesktop.org/~david/gudev/

so it's very easy to use libudev from G applications. The grand plan is
that the bulk of device enumeration / notification in GNOME will switch
to using either libudev (through GUdev) or things like DeviceKit-disks
and DeviceKit-power.

Of course, it will take some time until this bubbles into released
distros; I don't have any good advice on how to handle that other than
telling people to use newer distros. FWIW, what we're doing in GIO/GVfs
is that we have a soft dependency on things like libgudev / DeviceKit-*.
If people have new enough software they get a new backend, if they don't
(e.g. enterprise distros, version updates for enterprise/stable
distros), we fall back to existing old code. In practice it works out
fine.

> > So the next step should be to port Cheese to libudev, but as David
> > suggested [2], why not to do the detection directly in Gstreamer?
> > As far as I can tell the current option with Gstreamer is to use the
> > PropertyProbe interface.
> > But as it is at the moment it's suboptimal for our needs:
> > - the device detection code in v4l?src just tries to open
> > /dev/v4l/video* and /dev/video* devices. It doesn't even check the V4L
> > api they support.
> > Wouldn't be better to use libudev to enumerate all the devices from
> > the video4linux subsystem?
> >  
> I commented in the bug, maybe we should use this for all kind of
> hardware sources and sinks.

Sorry, what bug is that?

     David



------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises
looking to deploy the next generation of Solaris that includes the latest
innovations from Sun and the OpenSource community. Download a copy and
enjoy capabilities such as Networking, Storage and Virtualization.
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Filippo]

On Mon, Jun 1, 2009 at 11:11 PM, David Zeuthen <[hidden email]> wrote:
> Sorry, what bug is that?

http://bugzilla.gnome.org/show_bug.cgi?id=583640


Filippo

------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises
looking to deploy the next generation of Solaris that includes the latest
innovations from Sun and the OpenSource community. Download a copy and
enjoy capabilities such as Networking, Storage and Virtualization.
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Filippo]

On Mon, Jun 1, 2009 at 1:05 PM, Stefan Kost <[hidden email]> wrote:

> Filippo Argiolas schrieb:
>> - there seems to be no way to get video device capabilities without
>> starting a pipeline, both v4lsrc and v4l2src implement (and not in a
>> good way) propertyprobe just for the "device" property. Would it be
>> possible to use propertyprobe for "flags" property too? is it possible
>> to fix a property (i.e. the device) and probe another one (i.e. the
>> flags)?
>> What I'd like to do is:
>> pspec = gst_property_probe_get_property (probe, "device");
>> GValueArray *array = gst_property_probe_probe_and_get_values (probe, pspec);
>> for (i = 0; i < array->n_values; i++) {
>>      g_object_set_property (G_OBJECT (element), "device", device);
>>      pspec = gst_property_probe_get_property (probe, "flags");
>>      /* get flags for the current device */
>>      ...
>> }
>> Linking and playing a simple pipeline for each device just for exclude
>> radios and tuners is not exactly optimal from my point of view.
>>
> I don't think it has to be played. You probably not even need to link it
> as READY is enough for getting it to open the device.

Nope, READY is not enough because v4l2src (probably v4lsrc too, didn't
check) uses basesrc start vmethod to open the device and this happens
in READY->PAUSED so you can't get caps nor flags nor other properties
that need an open device just setting v4l2src to READY. Other sources
(at least pulsesrc, but probably others too) manually listen to state
changes.

>> - there is no way to monitor new devices attached. At the moment we
>> don't use such a thing but wouldn't be bad to have it. But does it fit
>> in gstreamer purpose?
>>
> I don't think so, I use HAL in buzztard to detect devices. There is
> nothing in gstreamer that is guaranteed to always run.

Maybe we could start a mainloop in v4l2src? not sure how good it would
work though... maybe a custom element that runs an udev_monitor and
posts new devices in the bus?

Filippo

------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises
looking to deploy the next generation of Solaris that includes the latest
innovations from Sun and the OpenSource community. Download a copy and
enjoy capabilities such as Networking, Storage and Virtualization.
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel