TextureView

The TextureView class is a view object that combines a view with a SurfaceTexture.

Render with OpenGL ES

A TextureView object wraps a SurfaceTexture, responding to callbacks and acquiring new buffers. When a TextureView object acquires new buffers, TextureView issues a view invalidate request and draws using the contents of the newest buffer as its data source, rendering wherever and however the view state indicates it should.

OpenGL ES (GLES) can render on a TextureView object by passing the SurfaceTexture to the EGL creation call, but this creates a problem. When GLES renders on a TextureView, BufferQueue producers and consumers are in the same thread, which can cause the buffer swap call to stall or fail. For example, if a producer submits several buffers in quick succession from the UI thread, the EGL buffer swap call needs to dequeue a buffer from the BufferQueue. However, because the consumer and producer are on the same thread, there won't be any buffers available and the swap call hangs or fails.

To prevent buffer swap stalls, BufferQueue always needs a buffer available for dequeuing. To do this, BufferQueue discards the contents of the previously acquired buffer when a new buffer is queued. It also places restrictions on minimum and maximum buffer counts to prevent a consumer from consuming all buffers at once.

Choose SurfaceView or TextureView

Note: In API 24 and higher, it's recommended to implement SurfaceView instead of TextureView.

SurfaceView and TextureView fill similar roles and are both citizens of the view hierarchy. However, SurfaceView and TextureView have different implementations. A SurfaceView object takes the same parameters as other views, but SurfaceView contents are transparent when rendered.

TextureView has better alpha and rotation handling than SurfaceView, but SurfaceView has performance advantages when compositing UI elements layered over videos. When a client renders with a SurfaceView object, the SurfaceView provides the client with a separate composition layer. SurfaceFlinger composes the separate layer as a hardware overlay if supported by the device. When a client renders with TextureView, the UI toolkit composites the TextureView object's content into the view hierarchy with the GPU. Updates to the content may cause other view elements to redraw, for example, if the other views are positioned on top of TextureView. After view rendering completes, SurfaceFlinger composites the app UI layer and all other layers, so that every visible pixel is composited twice.

Note: DRM-protected video can be presented only on an overlay plane. Video players that support protected content must be implemented with SurfaceView.

Case Study: Grafika's Play Video

Grafika's Play Video includes a pair of video players, one implemented with TextureView and one implemented with SurfaceView. The video decoding portion of the activity sends frames from MediaCodec to a surface for both TextureView and SurfaceView. The biggest difference between the implementations are the steps required to present the correct aspect ratio.

Scaling SurfaceView requires a custom implementation of FrameLayout. WindowManager needs to send a new window position and new size values to SurfaceFlinger. Scaling a TextureView object's SurfaceTexture requires configuring a transformation matrix with TextureView#setTransform().

After presenting the correct aspect ratio, both implementations follow the same pattern. When SurfaceView or TextureView creates the surface, the app code enables playback. When a user taps play, it starts a video decoding thread, with the surface as the output target. After that, the app code doesn't do anything—composition and display are handled by SurfaceFlinger (for SurfaceView) or by TextureView.

Case Study: Grafika's Double Decode

Grafika's Double Decode demonstrates manipulation of the SurfaceTexture inside TextureView.

Grafika's Double Decode uses a pair of TextureView objects to show two videos playing side by side, simulating a video conferencing app. When the orientation of the screen changes and the activity restarts, the MediaCodec decoders don't stop, simulating playback of a real-time video stream. To improve efficiency, the client should keep the surface alive. The surface is a handle to the producer interface in the SurfaceTexture's BufferQueue. Because the TextureView object manages the SurfaceTexture, the client needs to keep the SurfaceTexture alive to keep the surface alive.

To keep the SurfaceTexture alive, Grafika's Double Decode obtains references to SurfaceTextures from the TextureView objects and saves them in a static field. Then, Grafika's Double Decode returns false from TextureView.SurfaceTextureListener#onSurfaceTextureDestroyed() to prevent the destruction of the SurfaceTexture. TextureView then passes a SurfaceTexture to onSurfaceTextureDestroyed() that can be maintained across the activity configuration change, which the client passes to the new TextureView object through setSurfaceTexture().

Separate threads drive each video decoder. Mediaserver sends buffers with decoded output to the SurfaceTextures, the BufferQueue consumers. The TextureView objects perform rendering and execute on the UI thread.

Implementing Grafika's Double Decode with SurfaceView is harder than implementing with TextureView because SurfaceView objects destroy surfaces during orientation changes. Additionally, using SurfaceView objects adds two layers, which isn't ideal because of the limitations on the number of overlays available on the hardware.