Device manufacturers can implement the CameraX OEM vendor library to expose device-specific effects to third-party developers through the CameraX extensions interface. CameraX defines the CameraX extensions interface for vendor-implemented classes loaded at runtime. This page describes how to implement the OEM vendor library and enable it on devices.
Before implementing the vendor library, make sure you understand how the CameraX Jetpack support library works. To learn more about CameraX, see CameraX overview. For more information on vendor extensions, see Vendor extensions.
Architecture
Figure 1. Vendor extensions architecture diagram
This diagram describes the architecture of the CameraX vendor extensions. Third-party apps are built against the CameraX extensions library (camera-extensions) and use the camera extensions public API (camera-extensions API). The camera-extensions API is defined by CameraX and is updated when a new version of the camera-extensions library is released. The versioning of the camera-extensions public API and camera-extensions library are the same.
The extensions interface (extensions-interface) is defined by CameraX and allows the camera-extensions library to talk to the OEM vendor library (camera-extensions-stub). The OEM library must have an implementation that matches the extensions interface version. A single implementation of the camera-extensions library can support multiple versions of the extensions interface.
Implementing the OEM vendor library
These implementation instructions use the bokeh (portrait) vendor extension as
an example, but you can apply these to other extensions such as the HDR, beauty,
and night mode extensions. To do this, copy and paste the code used for the
bokeh extension and replace the extension name with the desired extension (for
example, replacing BokehImageCaptureExtenderImpl
with
HdrImageCaptureExtenderImpl
).
You aren't required to provide an implementation for every effect. An effect without a vendor implementation defaults to the CameraX implementation. If an effect isn't available, the default implementation doesn't enable the effect and reports to the third-party developer that the effect is unavailable.
Version verification
When loading the OEM library, CameraX verifies that the OEM library contains a version that's compatible with the extensions interface version (referred to as extension-version in this document). To determine version compatibility, CameraX checks only the major and minor versions (for example, 1.0) but doesn't check the patch version because that's used only for bug fixes, not interface changes. The required extension-version for specific APIs is noted in this document. APIs without a specified extension-version are compatible with extension-version 1.0.
To verify the version, CameraX queries the
ExtensionVersionImpl
interface. CameraX then uses the version reported by the OEM library to
determine the functionality that can be called.
Initialization
CameraX starts the initialization process when it has determined the version of
the extensions interface implemented by the OEM library. The
InitializerImpl.init
method signals to the OEM library that an app wants to
use extensions. No other calls to the OEM library (aside from version checking)
are made until OnExtensionsInitializedCallback
is called with a success
state.
This method must be implemented as of extension-version 1.1. For details, see
the source at
InitializerImpl
.
ExtenderStateListener interface
CameraX provides hooks into several places in its pipeline to allow the OEM
library to set relevant SessionParameters
and CaptureRequest
values. To
allow the OEM library to set these values at the specified times, implement the
ExtenderStateListener
interface. This interface must be implemented as part of any extender, whether
it's a preview, image capture, bokeh, or HDR extender.
Image capture
To support the extension for
image capture,
implement the corresponding ImageCaptureExtender
interface (for example,
BokehImageCaptureExtender
or
HdrImageCaptureExtender
).
ImageCaptureExtender
includes interfaces required for extensions related to image capture.
The
CaptureProcessor
interface must be implemented for post processing to be done at the app layer.
It's not required if processing is done in the camera HAL.
The following diagram illustrates the image capture process flow.
Figure 2. Image capture flow diagram
Example: BokehImageCaptureExtenderImpl
To support the bokeh extension for image capture, implement the
BokehImageCaptureExtenderImpl
class in the androidx.camera.extensions.impl
package.
Preview
To support the extension for
preview,
implement the corresponding PreviewExtender
interface (for example,
BokehPreviewExtender
or
HdrPreviewExtender
).
The PreviewExtender
interface includes the interfaces required for
preview-related extensions.
For details, see the source at
PreviewExtender
.
Image processing for the preview extension can be performed in the camera HAL or
the app layer. This is determined by the value of ProcessorType
, which is
returned by PreviewExtenderImpl
.
If the PROCESSOR_TYPE_REQUEST_UPDATE_ONLY
type is returned, processing is
performed in the HAL through CaptureRequest
keys. If the
PROCESSOR_TYPE_IMAGE_PROCESSOR
type is returned, processing is performed in
the app layer by the
PreviewImageProcessorImpl
interface. This interface operates on an Image
and TotalCaptureResult
pair.
As of extensions interface version 1.1, only the YUV_420_888
image format must
be supported.
The following diagram illustrates the process flow for the preview vendor extension.
Figure 3. Preview flow diagram
Example: BokehPreviewExtenderImpl
To support bokeh for preview, implement the
BokehPreviewExtenderImpl
class in the androidx.camera.extensions.impl
package.
Reference implementation
For a reference OEM vendor library implementation, see /platform/frameworks/support/camera/integration-tests/extensionstestlib/. Note that this implementation performs passthroughs without actually implementing the effects.
Setting up the vendor library on a device
The OEM vendor library isn't built into an app but instead is loaded from the
device at runtime by CameraX. The <uses-library>
tag declares that the
androidx.camera.extensions.impl
library, which is defined in the
AndroidManifest.xml
file, is a dependency of CameraX and must be loaded at runtime. This allows
third-party apps using vendor extensions to automatically attempt to load the
OEM vendor library. The OEM library is marked as optional so apps can run on
devices that don't have the library on the device.
CameraX handles this behavior automatically when an app tries to use a vendor extension as long as the device manufacturer places the OEM library on the device so that it can be discovered by the app.
To set up the OEM library on a device, do the following:
- Add a permission file, which is required by the
<uses-library>
tag, using the following format:/etc/permissions/ANY_FILENAME.xml
. For example,/etc/permissions/camera_extensions.xml
. The files in this directory provide a mapping of the library named in<uses-library>
to the actual file path on the device. Use the example below to add the required information to the file.
name
must beandroidx.camera.extensions.impl
as that's the library that CameraX searches for.file
is the absolute path of the file that contains the extensions implementation (for example,/system/framework/androidx.camera.extensions.impl.jar
).
<?xml version="1.0" encoding="utf-8"?> <permissions> <library name="androidx.camera.extensions.impl" file="OEM_IMPLEMENTED_JAR" /> </permissions>
Validation
To test your implementation of the OEM vendor library, use the example app at
androidx-master-dev/camera/integration-tests/extensionstestapp/
,
which runs through various vendor extensions.
Frequently asked questions (FAQ)
Are there any restrictions on API levels?
Yes. This depends on the Android API feature set that's required by the OEM
vendor library implementation. For example,
ExtenderStateListener.onPresetSession()
uses the
SessionConfiguration.setSessionParameters()
call to set a baseline set of tags. This call is available only on API level
28 and higher. For details on specific interface methods, see the
API reference documentation.