media-workshop

media-workshop@linuxtv.org

113 discussions

[ANN] Media Mini-Summit Draft Agenda for March 26th
by Hans Verkuil 23 Mar '15

23 Mar '15

This is the draft agenda for the media mini-summit in San Jose on March 26th. Time: 9 AM to 5 PM (approximately) Room: TBC (Mauro, do you know this?) Attendees: Mauro Carvalho Chehab - mchehab(a)osg.samsung.com - Samsung Laurent Pinchart - laurent.pinchart(a)ideasonboard.com - Ideas on board Hans Verkuil - hverkuil(a)xs4all.nl - Cisco Mauro, do you have a better overview of who else will attend? Agenda: Times are approximate and will likely change. 9:00-9:15 Get everyone installed, laptops hooked up, etc. 9:15-9:30 Introduction 9:30-10:30 Media Controller support for DVB (Mauro): 1) dynamic creation/removal of pipelines 2) change media_entity_pipeline_start to also define the final entity 3) how to setup pipelines that also envolve audio and DRM 4) how to lock the media controller pipeline between enabling a pipeline and starting it, in order to avoid race conditions See this post for more detailed information: https://www.mail-archive.com/linux-media@vger.kernel.org/msg85910.html 10:30-10:45 Break 10:45-12:00 Continue discussion 12:00-13:00 Lunch (Mauro, do you have any idea whether there is a lunch organized, or if we are on our own?) 13:00-14:40 Continue discussion 14:40-15:00 Break 15:00-16:00 Subdev hotplug in the context of both FPGA dynamic reconfiguration and project Ara (http://www.projectara.com/) (Laurent). 16:00-17:00 Update on ongoing projects (Hans): - proposal for Android Camera v3-type requests (aka configuration stores) - work on colorspace improvements - vivid & v4l2-compliance improvements - removing duplicate subdev video ops and use pad ops instead - others? Most of the time will be spent on DVB and the MC. Based on past experience this likely will take some time to get a concensus. Comments are welcome! Regards, Hans

7 7

[ANN] Topics needed for the agenda
by Hans Verkuil 02 Mar '15

02 Mar '15

Hi all, Please reply to this post with topics that you want to discuss during the upcoming media summit in San Jose. I'll collect all topics and turn it into a proper agenda. We currently have a one-day summit, but if we have a lot of topics we might add a second. Several core developers will also be available on Friday, so that is an option. Thanks! Hans

3 3

[ANNOUNCE] media mini-summit on March, 26 in San Jose together with ELC
by Mauro Carvalho Chehab 19 Feb '15

19 Feb '15

As discussed on our IRC #v4l channels and on the media ML, most of the core developers will be again this year in San Jose - CA - USA for the Embedded Linux Conference. There are several subjects that we've been discussing those days that require a face to face meeting. So, We'll be doing a media mini-summit on March, 26 (Thursday) at the Marriott San Jose. This time, Linux Foundation will be handling the subscriptions for the event, via their registering site: https://www.regonline.com/register/login.aspx?eventID=1623927&MethodId=0&Ev… So, you need to register for an "add-on" option as shown below: [ ] Linux Media Summit: Please click here if you would like to attend Linux Media Summit. Media Summit is the premier forum to discuss the Linux multimedia development for webcams, audio and video streaming devices and analog/digital TV support at the Linux Kernel and its userspace APIs. Date: Thursday, March 26, 2015 9:00 AM - 6:00 PM (Pacific Time) If you're already subscribed to the event, you can login using the same URL, select your register and register for the Media Summit. I hope to see you there! Ah, as usual, we'll be using the media-workshop(a)linuxtv.org ML for specific discussions about that, so the ones interested on participate are requested to subscribe it, and to submit themes of interest to the mailing lists. Regards! Mauro

2 1

[ANNOUNCE] Media summit Report - October, 17-18 2014 - Düsseldorf
by Mauro Carvalho Chehab 21 Oct '14

21 Oct '14

This report is also available in html at: http://linuxtv.org/news.php?entry=2014-10-21.mchehab In the name of the organizing committee, I'd like to thank you all for being there with us during those two days. There were several good discussions that happened during the meeting, aimed to improve even more the Linux media subsystem. Also, it was agreed during the meeting that we'll try to reach an even broader audience on the next events. So we're counting with all you on our next year's event(s). Thanks! Mauro - Attendees, in alphabetical order: NAME - EMAIL - ORGANIZATION Brad Love - blove(a)hauppauge.com - Hauppauge Chris Kohn - christian.kohn(a)xilinx.com - Xilinx Friedrich Schwittay - FSchwittay(a)pctvsystems.com - PCTV Guennaddi Liakhovetski - g.liakhovetski(a)gmx.de - Intel Hans de Goede - hdegoede(a)redhat.com - Red hat Hans Verkuil - hverkuil(a)xs4all.nl - Cisco Henning Garbers - hgarbers(a)pctvsystems.com - PCTV Johannes Stezenbach - js(a)linuxtv.org - LinuxTV Julien Beraud - julien.beraud(a)parrot.com - Parrot (www.parrot.com) Kamil Debski - k.debski(a)samsung.com - Samsung Laurent Pinchart - laurent.pinchart(a)ideasonboard.com - Ideas on board Lucas Stach - l.stach(a)pengutronix.de - Pengutronix Mauro Carvalho Chehab - mchehab(a)osg.samsung.com - Samsung Mike Krufky - mkrufky(a)linuxtv.org - Samsung Nicolas Dufresne - nicolas.dufresne(a)collabora.com - Collabora Pawel Osciak - pawel(a)osciak.com - Google Peter Griffin - peter.griffin(a)linaro.org - Linaro/ST Philipp Zabel - p.zabel(a)pengutronix.de - Pengutronix Ralph Metzler - rjkm(a)metzlerbros.de - Meltzler Brothers Ricardo Ribalda Delgado - ricardo(a)qtec.com - Qtechnolgy A/S Sakari Ailus - sakari.ailus(a)linux.intel.com - Intel Thursday: ======== 1) Configuration stores - Hans Verkuil =================================== - Need to report which settings/controls were applied - Drivers needs to be able to report full/limited Android CameraHAL v3 support (full means full synchronization support) - We probably need a mechanism to be able to remove (free) config stores - Can we use APPLY(0) instead of CLOSE? - Per buffer or per frame configuration? Note: there is no notion of frame IDs yet in V4L2. - Would be nice to be able to disable APPLY, perhaps a flag for the core similar to HAS STORE 2) Android camera HAL v3 - Sakari Ailus ==================================== http://www.retiisi.org.uk/v4l2/foil/android-camera-hal-v3-v4l2.pdf - Capture requests queued lead to returned buffers in order, best effort to make per-buffer configuration apply correctly needed - Capture requests/returned buffers do not map directly to QBUF/DQBUF because multiple buffers in different formats may be produced from a single request. Some outputs could be implemented in software, e.g. multiple YUV outputs when hardware simply does not contain as many. - Should statistical data be returned in a separate video buffer queue? Might be associated via sequence number, probably separate frame ID better - 3A library input: frame metadata, depends on sensor, might be in-frame or out-of band. If possible, separate buffer queue prepared to get to it early. - HAL v3 needs Start Of Frame and End Of Frame events - If there is no hardware mechanism for synchronization, apply sensor settings as soon as possible after SOF and hope for the best. It is possible to verify the timing has been successful in the user space. - Sensor settings and ISP settings applied at different points in time, as sensor settings usally take effect on the following frame - Sensors may implement all or parts of the ISP themselves. It may not be feasible to implement hal v3 full profile for such sensors. - Start of Exposure events not to be implemented in the kernel. HAL must calculate the event time based on other available information, such as end of frame (or start of frame event) and other frame timing parameters. - Some metadata tags could be mapped to V4L2 (compound) controls, they are associated with a single capture request. Some metadata tags (statistics) are produced from video buffer queues. Other metadata tags originate from HAL. - Needs a guarantee that a single capture request will produce all buffers requested by it with the request's parameters, needs per-buffer configuration, associating buffers with a certain frame. - A working halv3 implementation for UVC (limited profile) would be nice to have - A working halv3 implementation on top of some hardware (full profile) is needed to test assumptions - ISP configuration may be implemented by controls or via an output video buffer queue - V4L2: independent queues (possibly multiple video device nodes), frame synchronization needed when queueing the buffers, drivers are expected to always return those buffers, possibly with an error flag, HALv3: multiple streams are synchronized via capture requests - Requests may contain buffers for every queue or only some queues. - All buffers captured as part of a request must originate from the same sensor frame. Other behaviour is not allowed. - All buffers have start of exposure timestamps (approximated by start of frame timestamp - exposure time) - Frames from different capture queues can be associated using sequence numbers. - ISPs are typically connected to a fast bus, and mostly it's possible to synchronise changing the configuration per frame reliably. Should this be extended to sensors? In that case, the sensor driver would need timing information from the ISP. Sensors are typically connected over a slow bus such as i2c so error handling would also be needed. The alternative is to leave sensor synchronisation up to the user space. - Further discussion - How are buffers and controls from different device nodes associated as a single request by the driver? - Applying parameters from sub-device nodes 3) DVB API improvements - Mauro Carvalho Chehab ============================================ - New ioctl needed for retrieving the available PLP (physical layer pipe)that can be read back from frontend's detection. Eventually get_property might be used, but it could be too big for that. - How to support devices with multiple demodulators that could have a common PES/SEC filter block dynamically shared between them? Currently, the API assumes that each demod has a fixed number of filters. - It is desired to add mmap() and DMABUF support. The best seems to use videobuf2 for buffer delivery, but how to avoid V4L2 dependencies? It seems it should reuse memops but not videobuf2-core, reimplement queue/dequeue, state handling, see what should be shared. - TBD: How to improve integration of DVB frontends, demuxes and CAM drivers with other media components via the media controller. Work plan: - Make a more detailed list of requirements for DVB - to be done by the DVB developers; - Send patches for PLP table retrival by userspace - Hauppauge/PCTV; - Send a proposal for DVB-C2 - Ralph Meltzler; - After the above steps is completed, we should discuss between the DVB developers about who will be doing what. 4) Media development process - Hans Verkuil ======================================== - Fixes are written for an upcoming release typically based on findings from rc1 - Fixes should get higher priority - Fixes must be upstreamed by rc3 - Patches that go to the fixes tree are not applied to master. This is because the fixes sent to Linus will reappear from upstream, eventually to master branch as well. - Use 'GIT FIXES' for urgent bug fixes (Hans will send V4L2 submitting patches text (after updating) to Pawel for wiki inclusion) - Consider github as git repo for better failover. - Co-locating with other conferences creates schedule conflicts - Too costly to move before/after conferences (bosses will only pay for the meeting, not for the conference) - Making the mini-summit open for all would be an interesting experiment. - But we shouldn't charge for the mini-summit. - Next year: kernel summit in South Korea, so the media mini-summit will be in Korea as well. 5) V4L2 raw codec API - Pawel Osciak ================================= - The current codec API is suitable for codecs which perform all the stages of encoding or decoding. - Newer codec hardware implementations expect the CPU to perform less computing intensive parts of the codec. - VA-API like model: codec split between user and kernel spaces - Some vendors provide closed user space codec libraries. Yuck. - Bit stream formats are vendor independent, so the parsing of the bit stream should be as well. - Processing must be performed per-slice. This allows lower latency. - Decoding to be implemented in libv4l. Multiple hardware devices may benefit from the stream parsing functionality this way. - GStreamer plugins already implements stream parsing. License? 6) Multiple timestamps - Ricardo Ribalda ===================================== - Some Industrial Machines require metadata about the state of the machine when the image is taken - When this data is small enough (~32 bits) and it is similar to a timestamp (ex. encoder data) we might use the timecode struct, already part of vb2_buffer - Sometimes the metadata can be much bigger. An example has been shown regarding an optical grader of potatoes. In this sytem, the spinoflex (conveyor belt with fingers) has a state of around 1000bits - This information does not fit the event data structure and share the same problems as the camera statistics, therefore common solutions should be used. - The proposal of creating a new ioctl for this (GET_META) has been generally rejected. Instead two options have been propossed depending of the size: - A) Configuration stores are an option as well. A single array control could be used for this. - B)Video buffers could be used. This is very much analogous to sensor metadata after all. - Laurent pointed out that A single interface to pass statistics to the user space is preferred, independently of the size of the statistics. The common consensous seems to be that: - If the data is transferred to the memory, video buffers are probably the right solution. - For 1000 bits, compound controls are very likely the best option. - Image statistics of tens or hundreds of kilobytes should use a video buffer queue. Work plan: - Hans will make sure that the configuration store can support this usercase - We will define a generic fourcc for statistics - Libv4l will be in charge of converting the vendor specific statistics into this generic format Friday: ====== 7) Runtime reconfiguration of pipelines - Chris & Laurent ====================================================== - Xilinx need to support adding any type of daughterboard to their FPGA pipeline: userspace needs to load a DT overlay dynamically for that daughterboard. Note that the Xilinx driver won't know which devices are on that daughterboard since it is the customer/thirdparty that makes it. - Partial reconfiguration: ideally only the IP blocks that will be replaced need to be removed in the MC: this is complex. - Can we make an initial implementation to just tear down everything and rebuild it with the new IP instead of the old? Less flexible, but much easier to do until we can change the MC on the fly. - Option: perhaps just disable entities instead of deleting them. But it's a bit of a hack, and we should just bite the bullet and add proper support. - Result: implement everything tear down first. Later work on how to delete entities on the fly. - Create a virtual MC device driver: what should be there? - Hans Verkuil - sensor, at least two DMA engines giving two different formats - Bayer support? - different input ports (csi/parallel), cropping/scaler block - configurable pipeline (including m2m device) - flash, lens 8) Hierarchical media devices - Philipp Zabel ========================================== - Philipp gave a presentation about complex media devices that could be better represented via hierarchical trees. The discussions about that happened together with the next topic. 9) Highly reconfigurable hardware - Julien Beraud ============================================== - 44 sub-devices connected with an interconnect. - As long as formats match, any sub-device could be connected to any other sub-device through a link. - The result is 44 * 44 links at worst. - A switch sub-device proposed as the solution to model the interconnect. The sub-devices are connected to the switch sub-devices through the hardware links that connect to the interconnect. - The switch would be controlled through new IOCTLs S_ROUTING and G_ROUTING. - Patches available: http://git.linuxtv.org/cgit.cgi/pinchartl/media.git/log/?h=xilinx-wip 10) libdvbv5 - Mauro Carvalho Chehab ================================ - Part of libv4l v1.6.0 - Most applications still use DVBv3 API. - Contains MPEG-TS decoders for program information. - Compilation options to choose different parts of the library to be built. Not everyone needs everything. - The user may want to use custom replacements for parts of the library, especially for parsing tables like EPG. - The library still needs the MPEG-TS parsers for channel parsing. If the MPEG-TS parser isn't part of the library, the functionality would need to be available elsewhere using the same interface. - libdvbv5 license right now GPL, agreement on license change to LGPL exists from all contributors. 11) DVB tuners - Mauro Carvalho Chehab ================================== - Explained the old hybrid model and the new model used by the tuners and the odds with the current approach - It was also explained that I2C MUX is a need, in order to help fixing the problems with suspend/resume inside the subsystem - Some concerns arrised if the new model actually works or not for analog TV, as currently there's no tuner using it for analog - It was agreed that the new model is the way to go, but it requires a carefully review, and changes should not be done on all tuners at once, but, instead, in steps 12) Other DVB issues - Mauro Carvalho Chehab ======================================== - Better information on device capabilities is needed in the user space, as, currently, there are very few bits available for DVB caps at fe_caps enum, used by FE_GET_INFO. Also, some standards can have lots of options (ISDB-T has 28), and reporting there to userspace using bits is actually not a good option. - The main issue with fe_caps is that, currently, there are only two bits left, and this is not enough to represent the current capabilities of modern delivery system's frontends. a a few examples of missing things that we want to be able to report, we have: - Ralph reminded that there are DVB-C/C2 devices that only support automatic bitrate detection, and he is even unsure if it is possible to retrieve the detected bitrate from such frontends. So, we need a way to allow "CAN_AUTO_SYMBOL_RATE". That's bad, as applications like w_scan may run several times over the same symbol rates, and may actually be detecting the same transponder several times. - Complex delivery systems like ISDB-T have some combinations that aren't very common, and some chipset providers decided to not implement (Mauro). For example, mb86a20s frontend doesn't support guard interval equal to 1/32. - The agreeded strategy is to write a new ioctl that, once the delivery system is set, it will return what are the valid values/ranges that a given frontend property can support. In order to optimize the drivers, the core could pre-fill it with all that it is supported by a given delivery system, letting the driver to override it, disabling the features that aren't supported. - Mkruky proposed to add a new ioctl (FE_GET_PROPS_INFO - or something like that) that would return all the valid values for a given property for the current delivery system. - Old DVBv3 APIs that weren't used anymore should be clearly documented as deprecated 13) Dead pixels: Ricardo Ribalda ============================ - Position of dead pixels + classification of dead pixels (dead/lazy/others) - Store this on a flash: common format? - No problem with the dead pixel proposal, but it needs to be used upstream somewhere before it can be used. Ricardo's driver is unlikely to be upstreamed. Not sure if adding it to vivid is sufficent. - Suggestion: omap3 should have it, try getting a beagleboard/bone + sensor and implementing it there. - sensor/module metadata: needs an RFC first. 14) Multiple selections - Ricardo Ribalda ===================================== - Hans's patch "videodev2.h: add v4l2_ctrl_selection compound control type" implements selection controls. This replaces the selection interface, and existing drivers should be converted. - Pad information missing from the RFC patchset. - Multiple selections will not be available on selections interface. - Hans stopped working on this, but welcomes someone else continuing with it.

2 2

[ANNOUNCE] FINAL DRAFT: Media mini-summit notes
by Mauro Carvalho Chehab 21 Oct '14

21 Oct '14

Hi all, This is the final draft for the media mini-summit notes. My plan is to send to the media ML and post on our week in the next couple hours or so. Attendees, Please review. Michael, Do you have Brad's email? Also, please forward for him to be able to review it too. Thanks! Mauro - Media summit October 2014 Düsseldorf Attendees, in alphabetical order: NAME - EMAIL - ORGANIZATION Brad Love - - Hauppauge Chris Kohn - christian.kohn(a)xilinx.com - Xilinx Friedrich Schwittay - FSchwittay(a)pctvsystems.com - PCTV Guennaddi Liakhovetski - g.liakhovetski(a)gmx.de - Intel Hans de Goede - hdegoede(a)redhat.com - Red hat Hans Verkuil - hverkuil(a)xs4all.nl - Cisco Henning Garbers - hgarbers(a)pctvsystems.com - PCTV Johannes Stezenbach - js(a)linuxtv.org - LinuxTV Julien Beraud - julien.beraud(a)parrot.com - Parrot (www.parrot.com) Kamil Debski - k.debski(a)samsung.com - Samsung Laurent Pinchart - laurent.pinchart(a)ideasonboard.com - Ideas on board Lucas Stach - l.stach(a)pengutronix.de - Pengutronix Mauro Carvalho Chehab - mchehab(a)osg.samsung.com - Samsung Mike Krufky - mkrufky(a)linuxtv.org - Samsung Nicolas Dufresne - nicolas.dufresne(a)collabora.com - Collabora Pawel Osciak - pawel(a)osciak.com - Google Peter Griffin - peter.griffin(a)linaro.org - Linaro/ST Philipp Zabel - p.zabel(a)pengutronix.de - Pengutronix Ralph Metzler - rjkm(a)metzlerbros.de - Meltzler Brothers Ricardo Ribalda Delgado - ricardo(a)qtec.com - Qtechnolgy A/S Sakari Ailus - sakari.ailus(a)linux.intel.com - Intel Thursday: 1) Configuration stores - Hans Verkuil - Need to report which settings/controls were applied - Drivers needs to be able to report full/limited Android CameraHAL v3 support (full means full synchronization support) - We probably need a mechanism to be able to remove (free) config stores - Can we use APPLY(0) instead of CLOSE? - Per buffer or per frame configuration? Note: there is no notion of frame IDs yet in V4L2. - Would be nice to be able to disable APPLY, perhaps a flag for the core similar to HAS STORE 2) Android camera HAL v3 - Sakari Ailus http://www.retiisi.org.uk/v4l2/foil/android-camera-hal-v3-v4l2.pdf - Capture requests queued lead to returned buffers in order, best effort to make per-buffer configuration apply correctly needed - Capture requests/returned buffers do not map directly to QBUF/DQBUF because multiple buffers in different formats may be produced from a single request. Some outputs could be implemented in software, e.g. multiple YUV outputs when hardware simply does not contain as many. - Should statistical data be returned in a separate video buffer queue? Might be associated via sequence number, probably separate frame ID better - 3A library input: frame metadata, depends on sensor, might be in-frame or out-of band. If possible, separate buffer queue prepared to get to it early. - HAL v3 needs Start Of Frame and End Of Frame events - If there is no hardware mechanism for synchronization, apply sensor settings as soon as possible after SOF and hope for the best. It is possible to verify the timing has been successful in the user space. - Sensor settings and ISP settings applied at different points in time, as sensor settings usally take effect on the following frame - Sensors may implement all or parts of the ISP themselves. It may not be feasible to implement hal v3 full profile for such sensors. - Start of Exposure events not to be implemented in the kernel. HAL must calculate the event time based on other available information, such as end of frame (or start of frame event) and other frame timing parameters. - Some metadata tags could be mapped to V4L2 (compound) controls, they are associated with a single capture request. Some metadata tags (statistics) are produced from video buffer queues. Other metadata tags originate from HAL. - Needs a guarantee that a single capture request will produce all buffers requested by it with the request's parameters, needs per-buffer configuration, associating buffers with a certain frame. - A working halv3 implementation for UVC (limited profile) would be nice to have - A working halv3 implementation on top of some hardware (full profile) is needed to test assumptions - ISP configuration may be implemented by controls or via an output video buffer queue - V4L2: independent queues (possibly multiple video device nodes), frame synchronization needed when queueing the buffers, drivers are expected to always return those buffers, possibly with an error flag, HALv3: multiple streams are synchronized via capture requests - Requests may contain buffers for every queue or only some queues. - All buffers captured as part of a request must originate from the same sensor frame. Other behaviour is not allowed. - All buffers have start of exposure timestamps (approximated by start of frame timestamp - exposure time) - Frames from different capture queues can be associated using sequence numbers. - ISPs are typically connected to a fast bus, and mostly it's possible to synchronise changing the configuration per frame reliably. Should this be extended to sensors? In that case, the sensor driver would need timing information from the ISP. Sensors are typically connected over a slow bus such as i2c so error handling would also be needed. The alternative is to leave sensor synchronisation up to the user space. - Further discussion - How are buffers and controls from different device nodes associated as a single request by the driver? - Applying parameters from sub-device nodes 3) DVB API improvements - Mauro Carvalho Chehab - New ioctl needed for retrieving the available PLP (physical layer pipe)that can be read back from frontend's detection. Eventually get_property might be used, but it could be too big for that. - How to support devices with multiple demodulators that could have a common PES/SEC filter block dynamically shared between them? Currently, the API assumes that each demod has a fixed number of filters. - It is desired to add mmap() and DMABUF support. The best seems to use videobuf2 for buffer delivery, but how to avoid V4L2 dependencies? It seems it should reuse memops but not videobuf2-core, reimplement queue/dequeue, state handling, see what should be shared. - TBD: How to improve integration of DVB frontends, demuxes and CAM drivers with other media components via the media controller. Work plan: - Make a more detailed list of requirements for DVB - to be done by the DVB developers; - Send patches for PLP table retrival by userspace - Hauppauge/PCTV; - Send a proposal for DVB-C2 - Ralph Meltzler; - After the above steps is completed, we should discuss between the DVB developers about who will be doing what. 4) Media development process - Hans Verkuil - Fixes are written for an upcoming release typically based on findings from rc1 - Fixes should get higher priority - Fixes must be upstreamed by rc3 - Patches that go to the fixes tree are not applied to master. This is because the fixes sent to Linus will reappear from upstream, eventually to master branch as well. - Use 'GIT FIXES' for urgent bug fixes (Hans will send V4L2 submitting patches text (after updating) to Pawel for wiki inclusion) - Consider github as git repo for better failover. - Co-locating with other conferences creates schedule conflicts - Too costly to move before/after conferences (bosses will only pay for the meeting, not for the conference) - Making the mini-summit open for all would be an interesting experiment. - But we shouldn't charge for the mini-summit. - Next year: kernel summit in South Korea, so the media mini-summit will be in Korea as well. 5) V4L2 raw codec API - Pawel Osciak - The current codec API is suitable for codecs which perform all the stages of encoding or decoding. - Newer codec hardware implementations expect the CPU to perform less computing intensive parts of the codec. - VA-API like model: codec split between user and kernel spaces - Some vendors provide closed user space codec libraries. Yuck. - Bit stream formats are vendor independent, so the parsing of the bit stream should be as well. - Processing must be performed per-slice. This allows lower latency. - Decoding to be implemented in libv4l. Multiple hardware devices may benefit from the stream parsing functionality this way. - GStreamer plugins already implements stream parsing. License? 6) Multiple timestamps - Ricardo Ribalda -Some Industrial Machines require metadata about the state of the machine when the image is taken -When this data is small enough (~32 bits) and it is similar to a timestamp (ex. encoder data) we might use the timecode struct, already part of vb2_buffer -Sometimes the metadata can be much bigger. An example has been shown regarding an optical grader of potatoes. In this sytem, the spinoflex (conveyor belt with fingers) has a state of around 1000bits -This information does not fit the event data structure and share the same problems as the camera statistics, therefore common solutions should be used. -The proposal of creating a new ioctl for this (GET_META) has been generally rejected. Instead two options have been propossed depending of the size: -A) Configuration stores are an option as well. A single array control could be used for this. -B)Video buffers could be used. This is very much analogous to sensor metadata after all. -Laurent pointed out that A single interface to pass statistics to the user space is preferred, independently of the size of the statistics. The common consensous seems to be that: - If the data is transferred to the memory, video buffers are probably the right solution. - For 1000 bits, compound controls are very likely the best option. - Image statistics of tens or hundreds of kilobytes should use a video buffer queue. Work plan: - Hans will make sure that the configuration store can support this usercase - We will define a generic fourcc for statistics - Libv4l will be in charge of converting the vendor specific statistics into this generic format Friday: 7) Runtime reconfiguration of pipelines - Chris & Laurent - Xilinx need to support adding any type of daughterboard to their FPGA pipeline: userspace needs to load a DT overlay dynamically for that daughterboard. Note that the Xilinx driver won't know which devices are on that daughterboard since it is the customer/thirdparty that makes it. - Partial reconfiguration: ideally only the IP blocks that will be replaced need to be removed in the MC: this is complex. - Can we make an initial implementation to just tear down everything and rebuild it with the new IP instead of the old? Less flexible, but much easier to do until we can change the MC on the fly. - Option: perhaps just disable entities instead of deleting them. But it's a bit of a hack, and we should just bite the bullet and add proper support. - Result: implement everything tear down first. Later work on how to delete entities on the fly. - Create a virtual MC device driver: what should be there? - Hans Verkuil - sensor, at least two DMA engines giving two different formats - Bayer support? - different input ports (csi/parallel), cropping/scaler block - configurable pipeline (including m2m device) - flash, lens 8) Hierarchical media devices - Philipp Zabel - Philipp gave a presentation about complex media devices that could be better represented via hierarchical trees. The discussions about that happened together with the next topic. 9) Highly reconfigurable hardware - Julien Beraud - 44 sub-devices connected with an interconnect. - As long as formats match, any sub-device could be connected to any other sub-device through a link. - The result is 44 * 44 links at worst. - A switch sub-device proposed as the solution to model the interconnect. The sub-devices are connected to the switch sub-devices through the hardware links that connect to the interconnect. - The switch would be controlled through new IOCTLs S_ROUTING and G_ROUTING. - Patches available: http://git.linuxtv.org/cgit.cgi/pinchartl/media.git/log/?h=xilinx-wip 10) libdvbv5 - Mauro Carvalho Chehab - Part of libv4l v1.6.0 - Most applications still use DVBv3 API. - Contains MPEG-TS decoders for program information. - Compilation options to choose different parts of the library to be built. Not everyone needs everything. - The user may want to use custom replacements for parts of the library, especially for parsing tables like EPG. - The library still needs the MPEG-TS parsers for channel parsing. If the MPEG-TS parser isn't part of the library, the functionality would need to be available elsewhere using the same interface. - libdvbv5 license right now GPL, agreement on license change to LGPL exists from all contributors. 11) DVB tuners - Mauro Carvalho Chehab - Explained the old hybrid model and the new model used by the tuners and the odds with the current approach - It was also explained that I2C MUX is a need, in order to help fixing the problems with suspend/resume inside the subsystem - Some concerns arrised if the new model actually works or not for analog TV, as currently there's no tuner using it for analog - It was agreed that the new model is the way to go, but it requires a carefully review, and changes should not be done on all tuners at once, but, instead, in steps 12) Other DVB issues - Mauro Carvalho Chehab - Better information on device capabilities is needed in the user space, as, currently, there are very few bits available for DVB caps at fe_caps enum, used by FE_GET_INFO. Also, some standards can have lots of options (ISDB-T has 28), and reporting there to userspace using bits is actually not a good option. - The main issue with fe_caps is that, currently, there are only two bits left, and this is not enough to represent the current capabilities of modern delivery system's frontends. a a few examples of missing things that we want to be able to report, we have: - Ralph reminded that there are DVB-C/C2 devices that only support automatic bitrate detection, and he is even unsure if it is possible to retrieve the detected bitrate from such frontends. So, we need a way to allow "CAN_AUTO_SYMBOL_RATE". That's bad, as applications like w_scan may run several times over the same symbol rates, and may actually be detecting the same transponder several times. - Complex delivery systems like ISDB-T have some combinations that aren't very common, and some chipset providers decided to not implement (Mauro). For example, mb86a20s frontend doesn't support guard interval equal to 1/32. - The agreeded strategy is to write a new ioctl that, once the delivery system is set, it will return what are the valid values/ranges that a given frontend property can support. In order to optimize the drivers, the core could pre-fill it with all that it is supported by a given delivery system, letting the driver to override it, disabling the features that aren't supported. - Mkruky proposed to add a new ioctl (FE_GET_PROPS_INFO - or something like that) that would return all the valid values for a given property for the current delivery system. - Old DVBv3 APIs that weren't used anymore should be clearly documented as deprecated 13) Dead pixels: Ricardo Ribalda - Position of dead pixels + classification of dead pixels (dead/lazy/others) - Store this on a flash: common format? - No problem with the dead pixel proposal, but it needs to be used upstream somewhere before it can be used. Ricardo's driver is unlikely to be upstreamed. Not sure if adding it to vivid is sufficent. - Suggestion: omap3 should have it, try getting a beagleboard/bone + sensor and implementing it there. - sensor/module metadata: needs an RFC first. 14) Multiple selections - Ricardo Ribalda - Hans's patch "videodev2.h: add v4l2_ctrl_selection compound control type" implements selection controls. This replaces the selection interface, and existing drivers should be converted. - Pad information missing from the RFC patchset. - Multiple selections will not be available on selections interface. - Hans stopped working on this, but welcomes someone else continuing with it.

1 0

Etherpad used on the mini summit
by Ricardo Ribalda Delgado 21 Oct '14

21 Oct '14

https://etherpad.fr/p/media-summit-2014 -- Ricardo Ribalda

3 2

Fw: Deutsche Bahn Train Strike
by Mauro Carvalho Chehab 17 Oct '14

17 Oct '14

For the those that still in Germany, some information about the train strike and alternative transportation. Regards, Mauro Forwarded message: Date: Fri, 17 Oct 2014 05:06:50 -0700 From: Stephanie Markowski <smarkowski(a)linuxfoundation.org> To: Mauro Carvalho Chehab <m.chehab(a)samsung.com> Subject: Deutsche Bahn Train Strike Hi Mauro, I wanted to make sure you and your team were aware that the Deutsche Bahn Trains are going on strike. Below please find information on alternate transportation information: Please note that Deutsche Bahn trains are going on strike starting at 2am Saturday, October 18th and will remain on strike through Monday, October 20th. Should this affect your travel plans, below please find alternate transportation options. For more information, please visit: http://www.spiegel.de/reise/aktuell/bahnstreik-am-wochenende-das-muessen-si… CAR RENTAL There is a SIXT Car Rental Station at Düsseldorf Hauptbahnhof (Main Station) Address: Willi-Becker-Allee 10 40227 Düsseldorf GDS Code: DUSX02 Sixt-Station: 177 Opening Hours: Monday-Friday: 7am-9pm Saturday: 8am-3pm Sunday: 12pm-8pm BUS OPTIONS Flixbus: www.flixbus.de (running daily to Frankfurt Airport, prices from 8 euros) Mein Fernbus: www.meinfernbus.de (running daily to Frankfurt Main Station, prices from 22 euros) ADAC Bus: www.adac-postbus.de THE FOLLOWING TRANSPORTATION IS NOT AFFECTED BY THE STRIKE: U-BAHN BUSES AND TRAINS IF YOU ARE RIDING ON A TRAIN OPERATED BY ANOTHER COMPANY, PLEASE CHECK WITH THE TRAIN COMPANY TO MAKE SURE YOUR TRAVEL WILL NOT BE AFFECTED If you have any questions or need any additional information, please don't hesitate to contact us via email at events(a)linuxfoundation.org or visit us on the ground floor at registration. Thank you, LF Events -- Stephanie Markowski Event Logistics Manager The Linux Foundation T: 610-348-4600 F: 415-738-5434 E: smarkowski(a)linuxfoundation.org www.linuxfoundation.org Cheers, Mauro

1 0

[ANN] Room 11 for the media summit
by Hans Verkuil 14 Oct '14

14 Oct '14

Hi all, I checked and we have room 11 for the media summit on Thursday and Friday. See you there! Hans

2 2

[ANN] First tentative agenda
by Hans Verkuil 13 Oct '14

13 Oct '14

Hi all, I have collected all suggested topics. If I missed any, then let me know. There seem to be three groups of topics: one has general topics, one is related to the compound controls, per frame configuration and Android, and one topic is about complex video pipelines. Note that there are currently no DVB topics at all. I have grouped the suggestions into those three high-level topics. If you disagree with where I put it, then let me know. Also let me know your estimate of how long you think the discussion will take. I know it can be hard to pick a time, but it tends to average out anyway. If you put up a topic, then you are expected to prepare for it, either by making a small presentation, or at least to think about what you want to say. We have only two days and I don't want to waste any time. Speaking for myself: on one of the two days I have to give a lightning talk in the gstreamer conference and I want to attend Nicolas' gstreamer presentation as well. I don't know the times and dates yet. If you have suggested a topic and you also have specific times you won't be available, then let me know as soon as you have the details. The plan is to start each day at 9 am and go on to 5 pm or so, depending on how things work out. Regards, Hans ============= General Topics ================== Hans Verkuil: - A presentation on colorspaces (Do *you* know what to put in the v4l2_pix_format colorspace field? And why should it matter?) 30 min - media development process: what works, what doesn't 10-15 min Ricardo: - multiple timestamps per buffer ============== Per-frame configuration/camera HAL v3/Compound control types ============ Ricardo: - multi selections and dead pixel api Hans Verkuil: - update on configure stores for the control framework 15 min Sakari: - Android camera HAL API v3 and what kind of requirements it brings to V4L2. Pawel: - Existing codec API ambiguities (does this belong to this topic?) - A proposal for a new codec API extension/mode for HW codecs that can't parse elementary streams Guennadi: - camera support in Android (does this belong to this topic?) =========== Complex video pipeline drivers topic =================== Hans Verkuil: - create virtual complex omap3-like driver: what is needed 15 min Laurent & Chris Kohn: - runtime reconfiguration of pipelines Philip Zabel: - Helping userspace to use mem2mem devices; clarification of encoder/decoder handling, clarification of format/size setting in case of dependencies between input and output formats, possibly broad categorisation of mem2mem devices (encoder, decoder, scaler, rotator, csc/filter, ...) - Hierarchical media devices - what if you have a lot of media entities and some of them are more closely related to each other than others. Julien Beraud: -Highly reconfigureable hardware devices and the possibility to create media-links and virtual subdevices in order to simplify userland vision. Nicolas Dufresne: - I'll like to discuss frame size enumeration for scaling m2m (e.g. fimc/gscaler and CUDA decoder).

10 21

Fwd: Presentation
by Hans Verkuil 13 Oct '14

13 Oct '14

Forwarding Ricardo's presentation to this mailinglist. The presentation is available here: http://hverkuil.home.xs4all.nl/.ricardo/Qtec%20presentation.pdf I tried to send it directly to the mailinglist, but the attachment was larger than the maximum allowed by the ml software. So I uploaded to my site and you can just download it there. Regards, Hans -------- Forwarded Message -------- Subject: Presentation Date: Mon, 13 Oct 2014 12:16:33 +0200 From: Ricardo Ribalda Delgado <ricardo.ribalda(a)gmail.com> To: Hans Verkuil <hverkuil(a)xs4all.nl>

2 1

← Newer
1
2
3
4
5
6
7
8
...
12
Older →

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

media-workshop