Device-side metrics collectors

When running device-side tests (for example, instrumentations, UI Automator tests), host-side collectors might not be ideal because it's difficult to synchronize metric collection to a test running on a device. For example, a screenshot taken asynchronously is likely to miss the wanted screen and be useless.

To meet these use cases, a device-side version of our collectors exists and can be used in any AndroidJUnitRunner instrumentation. BaseMetricListener can be implemented to automatically report metrics that are collected in a way fully compatible with the Tradefed reporting pipeline.

This library is decoupled from Tradefed itself and can be used without Tradefed.

If you're using the AndroidJUnitTest runner from Tradefed, you can specify the following command line option to have your collector running with your tests:

  --device-listeners android.device.collectors.ScreenshotListener

Caution: In order for the collector classes to be resolved at runtime, your instrumentation APK needs to statically include them by adding the following to your makefile:

  LOCAL_STATIC_JAVA_LIBRARIES += collector-device-lib

Implementation

When implementing on top of the base class BaseMetricListener, you can choose when you would like to collect your metrics during the lifecycle of the instrumentation:

• When a test run starts: onTestRunStart
• When a test case starts: onTestStart
• When a test case ends: onTestEnd
• When a test case fails: onTestFail
• When a test run ends: onTestRunEnd

Interaction

The collection of metrics on the device side is made synchronously to the instrumentation execution itself, and metrics are passed back to the instrumentation results and parsed by Tradefed to be reported as part of the invocation.