Google is committed to advancing racial equity for Black communities. See how.

Setting up CTS

To run CTS, first prepare your physical environment, your desktop machine, and the Android device you're using for testing.

Physical environment

Bluetooth LE beacons

If the device under test (DUT) supports Bluetooth LE, place at least three Bluetooth LE beacons within 5 meters of the DUT for Bluetooth LE scan testing. Those beacons don't need to be configured or emit anything specific, and can be any kind, including iBeacon, Eddystone, or even devices simulating BLE beacons.

Cameras

When running camera CTS, use normal lighting conditions with a test pattern chart (such as a checkerboard pattern). Place the test pattern chart according to the DUT's minimum focus distance to ensure that it isn't too close to the lens.

Point the camera sensors to a scene with sufficient lighting to allow the sensors under test to reach and remain at the maximum configured target frames per second (FPS) as specified in CONTROL_AE_TARGET_FPS_RANGE. This applies to all camera sensors reported by getCameraIdList as the test iterates over the listed devices and measures performance individually.

If the DUT supports external cameras, such as USB webcams, plug in an external camera when running CTS. Otherwise, the CTS tests fail.

GPS/GNSS

If the DUT supports the global positioning system/global navigation satellite system (GPS/GNSS) feature, provide a GPS/GNSS signal to the DUT at a suitable signal level for reception and GPS location calculation. The GPS portion must be compliant with ICD-GPS-200C. Otherwise, the GPS/GNSS signal can be of any kind, including a satellite simulator or a GPS/GNSS repeater of outdoor signals, or you can place the DUT close enough to a window such that it can directly receive enough GPS/GNSS signal.

Wi-Fi and IPv6

CTS tests require a Wi-Fi network that supports IPv6, has an internet connection, and can treat the DUT as an isolated client. An isolated client refers to a configuration where the DUT doesn't have visibility to the broadcast/multinetwork messages on that subnetwork. This occurs with a Wi-Fi AP configuration or by running the DUT on an isolated subnetwork without other devices being connected.

If you don't have access to a native IPv6 network, an IPv6 carrier network, or a VPN to pass some tests depending on IPv6, you can use a Wi-Fi access point and an IPv6 tunnel. See Wikipedia's list of IPv6 tunnel brokers.

Wi-Fi RTT

Android includes the Wi-Fi RTT API for a Wi-Fi round trip time (RTT) capability. This allows devices to measure their distance to access points with an accuracy of 1 to 2 meters, significantly increasing indoor location accuracy.

Refer to Wi-Fi RTT (IEEE 802.11mc) and Wi-Fi location: ranging with RTT for a list of recommended devices supporting Wi-Fi RTT.

The access points should be powered up, but don't require a network connection. Access points don't need to be next to the testing device but are recommended to be within 40 feet of the DUT. One access point is typically sufficient.

Desktop machine setup

ADB and AAPT

Before running the CTS, ensure that you have installed the recent versions of both Android Debug Bridge (adb) and Android Asset Packaging Tool (AAPT) and added the location of those tools to the system path of your machine.

To install ADB, download the Android SDK Tools package for your operating system, open it, and follow the instructions in the included README file. For troubleshooting information, see Installing the Stand-alone SDK Tools.

Ensure that adb and aapt are in your system path. The following command assumes that you've opened the package archive in your home directory:

export PATH=$PATH:$HOME/android-sdk-linux/build-tools/version

Java Development Kit for Ubuntu

Install the proper version of Java Development Kit (JDK).

  • For Android 11, install OpenJDK11.
  • For Android 9 and Android 10, install OpenJDK9.
  • For Android 7.0, 7.1, 8.0 and 8.1, install OpenJDK8.

For details, see the JDK requirements.

CTS files

Download and open the CTS packages from Compatibility Test Suite Downloads matching your devices' Android version and all the application binary interfaces (ABIs) that your devices support.

Download and open the latest version of the CTS media files.

Device detection

Follow the step to set up your system to detect your device.

Memory limit

You might want to increase the maximum memory available during test run in the cts-tradfed script. Refer to example CL for more information.

Android device setup

User builds

A compatible device is defined as a device with a user/release-key signed build. Your device should be running a system image based on the known to be compatible user build (Android 4.0 or higher) from Codenames, Tags, and Build Numbers.

First API level build property

Certain CTS requirements depend on the build that a device was originally shipped with. For example, devices that originally ship with earlier builds might be excluded from system requirements that apply to devices that ship with later builds.

To make this information available to CTS, device manufacturers could have defined the build-time property ro.product.first_api_level. The value of this property is the first API level that the device was commercially launched with.

The device manufacturers can reuse the common underlying implementation to launch a new product as an upgrade of an existing product in the same device group. The device manufacturers can optionally set the API level of the existing product to ro.product.first_api_level, so that upgrade requirements are applied for CTS and Treble/VTS.

The device manufacturers can add PRODUCT_PROPERTY_OVERRIDES into their device.mk file to set this property, as shown in the following example:

#ro.product.first_api_level indicates the first api level that the device has
been commercially launched on.
PRODUCT_PROPERTY_OVERRIDES +=\
ro.product.first_api_level=21

First API level for Android 9 or higher

For devices launched with Android 9 or higher, set the ro.product.first_api_level property to a valid value from Codenames, Tags, and Build Numbers.

First API level for Android 8.x or lower

For devices launched on Android 8.x or lower, unset (remove) the ro.product.first_api_level property for the first build of the product. For all subsequent builds, set ro.product.first_api_level to the correct API level value. This allows the property to correctly identify a new product and preserves information about the first API level of the product. If the flag is unset, Android assigns Build.VERSION.SDK_INT to ro.product.first_api_level.

CTS shim packages

Android 10 or higher includes a package format called APEX. To run CTS tests for APEX management APIs (such as updating to a new version or reporting active APEXes) you must preinstall a CtsShimApex package on a /system partition.

The APEX shim validation test verifies the implementation of CtsShimApex.

ro.apex.updatable requirements

  • If the ro.apex.updatable property is set to true, CtsShimApex is required for all devices that support APEX package management.

  • If the ro.apex.updatable property is missing or isn't set, CtsShimApex isn't required to be preinstalled on the device.

CtsShim preinstalls and preloads

Starting with Android 11, CtsShimApex contains two prebuilt apps (built from build source), which don't contain any code except for the manifest. CTS uses these apps to test privileges and permissions.

If the device doesn't support APEX package management (that is, the ro.apex.updatable property is missing or isn't set), or if the device is running version 10 or lower, the two prebuilt apps must be preinstalled in the system separately.

Device Version Preinstall (if APEX supported) Preload
ARM x86 ARM x86
Android 11 android11-arm-release
under /system/apex/com.android.apex.cts.shim.apex
android11-x86-release
under /system/apex/com.android.apex.cts.shim.apex
android11-arm-CtsShim.apk
under /system/app/CtsShimPrebuilt.apk

android11-arm-CtsShimPriv.apk
under /system/priv-app/CtsShimPrivPrebuilt.apk

android11-x86-CtsShim.apk
under /system/app/CtsShimPrebuilt.apk

android11-x86-CtsShimPriv.apk
under /system/priv-app/CtsShimPrivPrebuilt.apk

Android 10 android10-release
under /system/apex/com.android.apex.cts.shim.apex
android10-arm-CtsShim.apk
under /system/app/CtsShimPrebuilt.apk

android10-arm-CtsShimPriv.apk
under /system/priv-app/CtsShimPrivPrebuilt.apk

android10-x86-CtsShim.apk
under /system/app/CtsShimPrebuilt.apk

android10-x86-CtsShimPriv.apk
under /system/priv-app/CtsShimPrivPrebuilt.apk

Android 9, O, and O-MR1 N/A N/A arm-CtsShim.apk
under /system/app/CtsShimPrebuilt.apk

arm-CtsShimPriv.apk
under /system/priv-app/CtsShimPrivPrebuilt.apk

x86-CtsShim.apk
under /system/app/CtsShimPrebuilt.apk

x86-CtsShimPriv.apk
under /system/priv-app/CtsShimPrivPrebuilt.apk

To pass the tests, preload the apps into the appropriate directories on the system image without re-signing the apps.

Sample Applet

Android 9 introduced Open Mobile APIs. For devices that report more than one secure element, CTS adds test cases to validate the behavior of the Open Mobile APIs. These test cases require the one-time installation of a sample applet into the embedded Secure Element (eSE) of the DUT or into the SIM card used by the DUT. The eSE sample applet and the SIM sample applet can be found in AOSP.

See CTS Test for Secure Element for more detailed information on Open Mobile API test cases and Access Control test cases.

Storage requirements

The CTS media stress tests require video clips to be on external storage (/sdcard). Most of the clips are from Big Buck Bunny, which is copyrighted by the Blender Foundation under the Creative Commons Attribution 3.0 license.

The required space depends on the maximum video playback resolution supported by the device. See section 5 in the Android Compatibility Definition document for the platform version of the required resolutions).

Note that the video playback capabilities of the DUT are checked through the android.media.CamcorderProfile APIs for earlier versions of Android and the android.media.MediaCodecInfo.CodecCapabilities APIs from Android 5.0.

Here are the storage requirements by maximum video playback resolution:

  • 480x360: 98 MB
  • 720x480: 193 MB
  • 1280x720: 606 MB
  • 1920x1080: 1863 MB

Screen and storage

  • Any device that doesn't have an embedded screen needs to be connected to a screen.
  • If the device has a memory card slot, plug in an empty SD card. Use an SD card that supports ultra high speed (UHS) bus with SDHC or SDXC capacity or one with at least speed class 10 or higher to ensure that it can pass the CTS.
  • If the device has SIM card slots, plug an activated SIM card into each slot. If the device supports SMS, each SIM card should have its own number field populated.

Developer UICC

In order to run CTS carrier API tests, the device needs to have a SIM card with carrier privilege rules on it. See Preparing the UICC.

Android device configuration

  1. Factory data reset the device: Settings > Backup & reset > Factory data reset.
  2. Set your device's language to English (United States): Settings > Language & input > Language.
  3. Turn on the location setting if there's a GPS or Wi-Fi/cellular network feature on the device: Settings > Location > On.
  4. Connect to a Wi-Fi network that supports IPv6, can treat the DUT as an isolated client (see Physical environment above), and has an internet connection: Settings > Wi-Fi.
  5. Make sure that no lock pattern or password is set on the device: Settings > Security > Screen lock > None.
  6. Enable USB debugging on your device: Settings > Developer options > USB debugging.
  7. Set the time to 12-hour format: Settings > Date & time > Use 24-hour format > Off.
  8. Set the device to stay awake: Settings > Developer options > Stay Awake > On.
  9. In Android 5.x and 4.4.x only, set the device to allow mock locations: Settings > Developer options > Allow mock locations > On.
  10. In Android 4.2 or higher, turn off USB app verification: Settings > Developer options > Verify apps over USB > Off.
  11. Launch the browser and dismiss any startup/setup screen.
  12. Connect the desktop machine that will be used to test the device with a USB cable.

File installation

Install and configure helper apps on the device.

  1. Set up your device according to your CTS version:
    • CTS versions 2.1 R2 through 4.2 R4: Set up your device (or emulator) to run the accessibility tests with:
      adb install -r android-cts/repository/testcases/CtsDelegatingAccessibilityService.apk

      On the device, enable delegation: Settings > Accessibility > Accessibility > Delegating Accessibility Service.
    • CTS versions 6.x or lower: On devices that declare android.software.device_admin, set up your device to run the device administration test using:
      adb install -r android-cts/repository/testcases/CtsDeviceAdmin.apk

      In Settings > Security > Select device administrators, enable the two android.deviceadmin.cts.CtsDeviceAdminReceiver* device administrators. Ensure that android.deviceadmin.cts.CtsDeviceAdminDeactivatedReceiver and any other preloaded device administrators remain disabled.
  2. Copy the CTS media files to the device as follows:
    1. Navigate (cd) to the path where the media files are downloaded and unzipped.
    2. Change the file permissions:
      chmod u+x copy_media.sh
    3. Copy the necessary files:
      • To copy clips up to a resolution of 720x480, run:
        ./copy_media.sh 720x480
      • If you aren't sure of the maximum resolution, copy all of the files:
        ./copy_media.sh all
      • If there are multiple devices under adb, add the serial option (-s) of a specific device to the end. For example, to copy up to 720x480 to the device with serial 1234567, run:
        ./copy_media.sh 720x480 -s 1234567