Camera extensions validation tool

The camera extensions validation tool allows device manufacturers to verify that the Camera Extensions OEM vendor library is implemented correctly. The tool includes automated and manual validation tests.

  • Automated validation tests: validate that the vendor library interface is implemented correctly. For example, if CaptureProcessor is required for image capture, the tests validate that ImageCaptureExtenderImpl#getCaptureStages() returns the required CaptureStage instances for capturing the images.

  • Manual validation tests: Validate the image effects and quality of preview and captured images. For example, the tests allow device manufacturers to manually validate whether the face retouch effect is applied correctly or whether the bokeh strength is sufficient.

The source code of the validation tool is part of the extensions test app in the Android Jetpack repository.

Build the camera extensions validation tool

To build the extensions validation tool, follow these steps:

  1. Download the Android Jetpack library source code. For details, see the Checking out the Code section of the Android Jetpack README.

  2. Build the extensionstestapp APK. This allows for running manual validation tests.

      cd path/to/checkout/frameworks/support/
  ./gradlew camera:integration-tests:camera-testapp-extensions:assembleDebug

    The APK is output to the following path:

      path/to/checkout/out/androidx/camera/integration-tests/camera-testapp-extensions/build/outputs/apk/debug/camera-testapp-extensions-debug.apk

  3. Build the androidTest APK. This APK allows for running automated validation tests.

      cd path/to/checkout/frameworks/support/
  ./gradlew camera:integration-tests:camera-testapp-extensions:assembleAndroidTest

    The APK is output to the following path:

      path/to/checkout/out/androidx/camera/integration-tests/camera-testapp-extensions/build/outputs/apk/androidTest/debug/camera-testapp-extensions-debug-androidTest.apk

Run automated validation tests

To run the automated validation tests, install the extensionstestapp and androidTest APKs.

  • extensionstestapp APK

      adb install -r path/to/checkout/out/androidx/camera/integration-tests/camera-testapp-extensions/build/outputs/apk/debug/camera-testapp-extensions-debug.apk

  • androidTest APK

      adb install -r path/to/checkout/out/androidx/camera/integration-tests/camera-testapp-extensions/build/outputs/apk/androidTest/debug/camera-testapp-extensions-debug-androidTest.apk

Run all automated tests

After installing the APKs, to run all automated tests to validate the vendor library implementation, run the following command:

adb shell am instrument -w -r androidx.camera.integration.extensions.test/androidx.test.runner.AndroidJUnitRunner

If all tests pass, an OK result is returned. Otherwise, the final test report shows failures in the terminal after all tests are complete.

automated_validation_result-pass

Figure 1. Automated tests OK result

automated_validation_result-fail

Figure 2. Automated tests result with failures

Run automated tests of a specific class

To run automated tests of a specific class, specify the name and path of the target class. The following example shows the command to run tests for the ImageCaptureTest class:

adb shell am instrument -w -r -e class **androidx.camera.integration.extensions.ImageCaptureTest** androidx.camera.integration.extensions.test/androidx.test.runner.AndroidJUnitRunner

Run manual validation tests

The manual validation tests are found in the extensions test app. After installing and launching the extensions test app, switch to the validation tool mode by tapping the menu item at the top right.

After switching to the validation tool mode, the first page lists all cameras that contain the REQUEST_AVAILABLE_CAPABILITIES_BACKWARD_COMPATIBLE capability. If a camera doesn’t support any extension mode, its corresponding list item is gray.

camera_validation_result-initial

Figure 3. Validation tool mode

Tap one of the cameras to see the extension modes for testing. Extension modes not supported by the selected camera are shown in gray.

extension_mode_validation_result-initial

Figure 4. Extension modes available for a camera

Verify the preview results

To verify the preview results, start a manual test by tapping an extension mode for the selected camera. An image capturing activity containing the preview is then shown.

preview-bokeh_enabled

Figure 5. Preview image with bokeh enabled

The image capturing activity supports the following functions:

  • Zoom in/out
  • Tap-to-focus
  • Flash modes switch button
  • EV +/-
  • Extension enabled/disabled switch button

Verify that the zoom in/out, tap-to-focus, flash modes, and EV +/- functions work as expected in the preview.

Verify the captured image results

To capture the image, tap the CAPTURE button (the round button) in the image capturing activity. This launches an image validation activity that shows the captured image.

photo_viewer-bokeh_enabled

Figure 6. Captured image with bokeh enabled

The image validation activity includes the following functions:

  • Pinch in/out to scale the image
  • Slide left/right to switch the captured images
  • Recapture
  • Save image menu item

Verify whether the captured image is correct and matches the zoom in/out, tap-to-focus, flash modes, and EV +/- settings that were set when capturing the image.

If the captured results are correct, tap the PASS button (checkmark) in the bottom-right corner. Otherwise, tap the FAIL button (exclamation point) in the bottom-left corner.

View test results

After an extension mode is verified as a pass or a fail, the list item for the extension mode shows a different background color and indicator. In the list view of all the cameras, the items are displayed in the following colors:

  • White background: The camera supports at least one extension mode and the supported extension modes aren't validated completely.
  • Green background: The camera supports at least one extension mode. All supported extension modes are validated with all results passing.
  • Red background: The camera supports at least one extension mode. All supported extension modes are validated with at least one extension mode result failing.
  • Gray background: This feature is unavailable.

camera_validation_result extension_mode_validation_result

Figure 7. Colors indicating test results for cameras and extension modes

Other validation tool functions

After all the tests are complete, the camera list activity provides the following functions:

  • Export test results: Exports the test results as a CSV file to the Documents/ExtensionsValidation folder.
  • Reset: Clears all cached test results.
  • Extensions sample app: Switches to the extensions sample app mode.

After completing the testing, you can export the test results. If you find an issue and a new version of the vendor library implementation with the solutions needs to be verified again, reset the previous test results and rerun all the supported extension modes on all cameras to verify that the issues are fixed.