Android 15 introduces a standardized process for integrating region of interest (RoI) user configurations into the Android video encoding framework. This feature enables better compression quality for RoIs by offering public APIs to integrate and analyze user configurations into the video encoder.
Implementation
SoC vendors and OEMs can control RoI support for video encoding with the
FEATURE_Roi
flag. If FEATURE_Roi
isn't enabled, there is no change in the video encoder behavior.
Changes in video framework
This section details the changes in the video framework needed to implement this feature.
Keys in Codec2
In Android 15, the quantization parameter map
(QP_map
) and rectangular configuration (rect
) are defined as RoI
configuration types. Additionally, two keys are introduced in Codec2 (C2) to
support these new types. Usage for both the keys is the same as the public APIs.
Within the encoder's frame loop, the configuration is dynamically adjusted
during the running stage, prior to the input buffer being queued, described as
follows:
In a sticky scenario, if no RoI configuration is provided for the current frame, the encoder uses the same configuration as the previous frame.
In a dynamic scenario, the RoI configuration types can change dynamically.
The new keys in C2 are described in the following sections.
kParamIndexQpOffsetMapBuffer
The kParamIndexQpOffsetMapBuffer
key signals the qp-offset
map for a frame. Its value is set using the input
parameter PARAMETER_KEY_QP_OFFSET_MAP
from setParameters
.
kParamIndexQpOffsetMapBuffer
is a byte array in
C2InfoBuffer
,
with the following attributes:
Length: The number of 16x16 blocks in one frame.
Value: Each value of the array is the QP offset of a 16x16 block, in region -51~51. The QP of the target largest coding unit (LCU) is calculated by the encoder rate control plus the offset. If the calculated result exceeds the 0~51 region, the value is truncated to 0~51.
- If 0: No QP offset, the QP is decided by the original rate control.
- If nonzero: QP is the original rate control plus offset.
- If negative: The video quality gets boosted in the target LCU.
- If positive: The video quality decreases in the target LCU.
Usage: The user must configure this key as 16x16 blocks. The encoder adjusts the configuration to the real LCU size by averaging the values of the 16x16 blocks in the LCU.
C2_PARAMKEY_QP_OFFSET_RECTS
The C2_PARAMKEY_QP_OFFSET_RECTS
key (set to coding.qp-offset-rects
sets the RoI as QpOffset-Rects
. Its value is set using the input parameter PARAMETER_KEY_QP_OFFSET_RECTS
from setParameters
.
To support this key, the following structure C2QpOffsetRectStruct
is introduced:
struct C2QpOffsetRectStruct : C2Rect {
int32_t qpOffset;
DEFINE_AND_DESCRIBE_C2STRUCT(QpOffsetRect)
C2FIELD(width, "width")
C2FIELD(height, "height")
C2FIELD(left, "left")
C2FIELD(top, "top")
C2FIELD(qpOffset, "qp-offset")
}
Where:
top
andleft
: The coordinates of RoI, in a rectangular shape. The RoI is stretched out to align with LCU boundaries. The value represents the top-left corner of each pixel, such that such that ((0,0), (16, 16)) defines a full 16x16 block.qpOffset
: Each value of the array represents the QP offset of the targetrect
area. Its definition and usage are the same as that of thekParamIndexQpOffsetMapBuffer
value.
Mapping algorithm
The following table shows the mapping from the public keys to the video framework:
Public keys or APIs | Mapping in video framework |
---|---|
PARAMETER_KEY_QP_OFFSET_MAP |
Value is passed to kParamIndexQpOffsetMapBuffer as a
C2InfoBuffer instance. |
PARAMETER_KEY_QP_OFFSET_RECTS |
Value is converted from String to
Struct C2QpOffsetRectStruct and passed to
C2_PARAMKEY_QP_OFFSET_RECTS . |
Error handling
The OEM implementation must handle the following error cases:
Error case | Example | Handling |
---|---|---|
Both vendor key and standardized key are used to turn on RoI. | User calls both setFeatureEnabled(FEATURE_ROI) AND the
vendor key to turn on RoI. |
RoI must be turned on. |
QP offset is within range but not supported by SoC vendors. | User sets the QP offset as 12, but SoC only supports QP offset up to 10. | The supported QP offset range is left as best effort. The value is clamped to the SoC's supported range. |
Multiple RoI configurations (whether a standardized key or a vendor-specific key) are set to a single frame. | User uses both standardized key and vendor key for frame 1. | If available, the framework retains the first standardized rect
configuration, the first standardized QP_map configuration,
or both. In each category, the framework sends only one standardized
configuration to SoC and if the standardized configuration is available,
the SoC implementation must ignore the vendor configurations. If
multiple configurations are sent to the SoC, the SoC vendor must retain
only one RoI configuration and ignore the rest of the configurations.The configurations are retained in this priority order:
|