Agenda for the Media Workshop 2016 - San Diego

Attendees:

Mauro Carvalho Chehab mchehab@osg.samsung.com (Samsung) Lars-Peter Clausen lars@metafoo.de (Analog Devices) Magnus Damm magnus.damm@gmail.com (Renesas) Shuah Khan shuahkh@osg.samsung.com (Samsung) Guennadi Liakhovetski g.liakhovetski@gmx.de (Intel) Kuninori Morimoto kuninori.morimoto.gx@renesas.com (Renesas) Benoit Parrot bparrot@ti.com (Texas Instruments) Laurent Pinchart laurent.pinchart@ideasonboard.com (Ideas on Board) Niklas Söderland niklas.soderlund@ragnatech.se (Ragnatech) Hans Verkuil hverkuil@xs4all.nl (Cisco)

1. CEC Framework Status update (Hans Verkuil) 09:30-09:40

First version of the framework close to completion. Will likely be submitted for Kernel 4.7, although it could be merged only for 4.8. Independent framework, allows consumers in V4L2, DRM, ALSA, ...

Driver support WIP for Pulse Eight USB dongle, omap4, adv7604/7842/7511.

Future plans:

ARC/CDC hotplug support

Implement high level protocol constraints (resend, timeout, rate limiting of messages). Whether those constraints can be implemented in the kernel remains to be analyzed, as the kernel to userspace API might be at a much lower level than the constraints. A userspace library might be another option.

2. Quick demo of the new qdisp utility that is in development (Hans Verkuil) 09:40-09:50

qdisp is a simpler alternative to qv4l2 that handles video capture and show the captured video only.

qdisp requires OpenGL, OpenGLES is planned.

Color space and format conversion code is based on GPU shaders. It will be split into a library to be shared with qv4l2. A CPU-based alternative would be feasible but isn't planned at the moment.

qdisp is available here: http://git.linuxtv.org/hverkuil/v4l-utils.git/log/?h=cec

3. Request API Status update (Laurent Pinchart) 09:50-10:00

At the moment: Allows to chain multiple of the existing IOCTLs into a request which will either be applied atomicly or not at all

New IOCTL to start a commit

APPLY operation will apply changes in a request immediately. Useful to change multiple controls at the same time

QUEUE operation will queue changes with a buffer and will be applied when the buffer is processed

Alternative: One new IOCTL that takes all state and applies it atomicly (like DRM atomic modesetting)

Open question: How to perform atomic operations across subssytems? V4L2, DRM, ALSA, MC

Action item Hans: contact Pawel to see what/when needs to be upstreamed for e.g. the rockchip driver.

4. Stream multiplexing (Guennadi Liakhovetski) 10:15-11:00

CSI-2 has up to 4 virtual channels (2 bits), 6 bits for Data Type

Virtual channels do not have to be in sync with one another, so different virtual channels can carry different framerates.

Within a virtual channel each line is tagged with a data type as well, so it can be used to pass metadata + videodata in one virtual channel as well.

Requirements:

- Pipeline validation

- Format validation

- Bandwidth/QoS

- Routing (related to muxing/demuxing)

One idea: Introduce the concept of virtual channels which are routed ontop of the physical links. A virtual channel has a route that goes through multiple physical entities, with routing information at each entity on how the data is forwarded.

Sounds interesting to have a non-V4L2 specific solution to solve cases where there a multiple entities linked into a bus or needing switch.

TDM for ALSA

Action item Laurent: dig up old router entity code and post it or provide a link to that code.

5. DT Bindings for flash & lens controllers 11:00-11:15

There are drivers that create their MC topology using the device tree information, which works great for entities that transport data, but how to detect entities that don't transport data such as flash devices, focusers, etc.? How can those be deduced using the device tree? (Hans)

Sensor DT node add phandle to focus controller.: add generic v4l binding properties to reference such devices.

6. How to improve the linux-media patch and review process? 11:00-12:15

Action item Mauro: contact Kamil to ask status of codec sub-maintainership. If no time, then Hans can take over.

No interest in DVB/RC from other developers.

Shuah can help with Media Controller patch reviews

Idea from Daniel Vetter to handle public APIs that might need some tweaking is to have it depend on a debug config option so enabling that api would make the kernel tainted.

Action item Mauro: Push on Intel (Sakari, Guennadi) (perhaps talk to Dirk Hohndel), Samsung (Marek and team), Google (Pawel) to give them time for upstreaming/reviewing.

For rc maintainers: also post rfc to linux-input.

Should we organize the next media workshop at ELC-E or LPC/KS ? Let's try with a quick survey of who plans to go where. As this workshop is held in the US Europe would be good to attract more European developers.

7. Fix broken media_device alloc/remove - Media Device Allocator API (Shuah Khan) 12:15-13:00

https://drive.google.com/open?id=0B0NIL0BQg-AlTUQxdG1zWW15Tmc

Media module needs to be owner of the media devnode - not the first driver that registers it With the above change, media_devnode_register() and __media_device_register() don't need struct module passed in. This can be cleaned up. Setting cdev parent should be set correctly - reference iio-core

Media Device Allocator API and change all drivers to use it. Dynamically changing the graph in a two driver model isn't exposed to use space - (application needs to poll to find changes to the topology) - considered unsupported at the moment.

8. Connector Entity discussion (Mauro) 14:00-15:30

What are userspace needs that are not covered by our APIs today and that could benefit from the MC API ?

What device nodes are related and are to be used together to capture analog TV? This is unrelated to connectors.

Analog and digital paths are usually exclusive, without a way for userspace to know what the constraints are. The representation is logical connection based.

Present information about connectors to the user to make it easier to know where to plug cables. The representation is physical connector based.

A connection-based representation in MC would require properties to map them to physical connectors. A connector-based representation in MC would require properties to map them to logical connections. The two last use cases above can thus be implemented with an MC representation based on either logical inputs or physical connectors, they don't require one specific representation.

Problem is similar to virtual channels over a logical link (like CSI-2, see "4. Stream multiplexing"). Logical connectors can be thought of as a specific routing of one or more signals from the physical connector through some fabric (e.g. switch, crossbar) to the demodulator.

Idea: Model physical connectors and support a routing ioctl for the entities that they are connected to. For existing drivers that use S_INPUT, we can either not show the logical connectors at all, or we just show the logical connectors in the absence of knowledge about the actual physical connectors.

How that routing ioctl should look like is unknown, but this might be done the same way as the routing as discussed in the context of CSI-2.

Some subdevs that have complex routing are saa7115, msp34xx and adv7842.

Action items: - Code S-Video with 2 pads; - Onesink PAD per input at the analog demod; - RF and Connections are OK - just one pad; - For now, not map Composite over S-Video.

9. Pad identification (Mauro) 15:45-16:30

A pad is identified by index and whether it is a input or output pad. If the index changes between kernel versions the userspace ABI breaks.

Use an enum to give a type to the pad. Don't expose this to userspace yet.

Adding pad names exposed to userspace can also be useful. To avoid repeating the same mistake we did with subdev and entity names, the API should define in details what the name should contain and how it should be constructed. What userspace can expect from the name, including information that can be extracted from the name (e.g. can I sscanf("%u") to get the pad number ?) also needs to be defined in the API.

10. Anything else? 16:45-

When adding a new API that is still experimental, the DRM/KMS subsystem covers the API with a configuration option and taints the kernel when the option is selected. This lasts for a few kernel versions until the API stabilizes and is then removed. This could be experimented in V4L2, effectively giving us back CONFIG_EXPERIMENTAL.

Should we avoid reserved fields in new ioctls, or use API versioning (as in DRM/KMS) ? Versioning is a bit more work in userspace, but avoids issues with application that don't zero reserved fields and break when the API is extended.

11. Other action items

- Review MC virtual driver v3 patch (vimc) form Hellen - Submit/review subdev patch that dynamically allocate patches (Laurent) - Split off legacy defines in media.h to media-legacy.h (Hans) - User versioning for CEC ioctls instead of reserved fields (Hans)