OEM guidance for RoI implementation

Android 15 introduces a standardized process for integrating region of interest (RoI) user configurations into the Android video encoding framework. This feature enables better compression quality for RoIs by offering public APIs to integrate and analyze user configurations into the video encoder.

Implementation

SoC vendors and OEMs can control RoI support for video encoding with the FEATURE_Roi flag. If FEATURE_Roi isn't enabled, there is no change in the video encoder behavior.

Changes in video framework

This section details the changes in the video framework needed to implement this feature.

Keys in Codec2

In Android 15, the quantization parameter map (QP_map) and rectangular configuration (rect) are defined as RoI configuration types. Additionally, two keys are introduced in Codec2 (C2) to support these new types. Usage for both the keys is the same as the public APIs. Within the encoder's frame loop, the configuration is dynamically adjusted during the running stage, prior to the input buffer being queued, described as follows:

• In a sticky scenario, if no RoI configuration is provided for the current frame, the encoder uses the same configuration as the previous frame.

• In a dynamic scenario, the RoI configuration types can change dynamically.

The new keys in C2 are described in the following sections.

kParamIndexQpOffsetMapBuffer

The kParamIndexQpOffsetMapBuffer key signals the qp-offset map for a frame. Its value is set using the input parameter PARAMETER_KEY_QP_OFFSET_MAP from setParameters.

kParamIndexQpOffsetMapBuffer is a byte array in C2InfoBuffer, with the following attributes:

• Length: The number of 16x16 blocks in one frame.

• Value: Each value of the array is the QP offset of a 16x16 block, in region -51~51. The QP of the target largest coding unit (LCU) is calculated by the encoder rate control plus the offset. If the calculated result exceeds the 0~51 region, the value is truncated to 0~51.

  • If 0: No QP offset, the QP is decided by the original rate control.
  • If nonzero: QP is the original rate control plus offset.
  • If negative: The video quality gets boosted in the target LCU.
  • If positive: The video quality decreases in the target LCU.

• Usage: The user must configure this key as 16x16 blocks. The encoder adjusts the configuration to the real LCU size by averaging the values of the 16x16 blocks in the LCU.

C2_PARAMKEY_QP_OFFSET_RECTS

The C2_PARAMKEY_QP_OFFSET_RECTS key (set to coding.qp-offset-rects sets the RoI as QpOffset-Rects. Its value is set using the input parameter PARAMETER_KEY_QP_OFFSET_RECTS from setParameters.

To support this key, the following structure C2QpOffsetRectStruct is introduced:

struct C2QpOffsetRectStruct : C2Rect {
  int32_t qpOffset;

  DEFINE_AND_DESCRIBE_C2STRUCT(QpOffsetRect)
  C2FIELD(width, "width")
  C2FIELD(height, "height")
  C2FIELD(left, "left")
  C2FIELD(top, "top")
  C2FIELD(qpOffset, "qp-offset")
}

Where:

• top and left: The coordinates of RoI, in a rectangular shape. The RoI is stretched out to align with LCU boundaries. The value represents the top-left corner of each pixel, such that such that ((0,0), (16, 16)) defines a full 16x16 block.

• qpOffset: Each value of the array represents the QP offset of the target rect area. Its definition and usage are the same as that of the kParamIndexQpOffsetMapBuffer value.

Note: For Android 15, at the initial stage, the video framework passes a blank rect configuration to the encoder to signal that the RoI feature is turned on.

Mapping algorithm

The following table shows the mapping from the public keys to the video framework:

Public keys or APIs	Mapping in video framework
PARAMETER_KEY_QP_OFFSET_MAP	Value is passed to kParamIndexQpOffsetMapBuffer as a C2InfoBuffer instance.
PARAMETER_KEY_QP_OFFSET_RECTS	Value is converted from String to Struct C2QpOffsetRectStruct and passed to C2_PARAMKEY_QP_OFFSET_RECTS.

Error handling

The OEM implementation must handle the following error cases:

Error case	Example	Handling
Both vendor key and standardized key are used to turn on RoI.	User calls both setFeatureEnabled(FEATURE_ROI) AND the vendor key to turn on RoI.	RoI must be turned on.
QP offset is within range but not supported by SoC vendors.	User sets the QP offset as 12, but SoC only supports QP offset up to 10.	The supported QP offset range is left as best effort. The value is clamped to the SoC's supported range.
Multiple RoI configurations (whether a standardized key or a vendor-specific key) are set to a single frame.	User uses both standardized key and vendor key for frame 1.	If available, the framework retains the first standardized rect configuration, the first standardized QP_map configuration, or both. In each category, the framework sends only one standardized configuration to SoC and if the standardized configuration is available, the SoC implementation must ignore the vendor configurations. If multiple configurations are sent to the SoC, the SoC vendor must retain only one RoI configuration and ignore the rest of the configurations. The configurations are retained in this priority order: Standardized rect Standardized QP_map Vendor rect Vendor QP_map