Trade Federation is a continuous test framework for running tests on Android devices. To run the Compatibility Test Suite (CTS), first read the Trade Federation Overview for an explanation of Tradefed test framework.
To run a test plan:
- Set up Devices Under Test (DUTs) as per Android device setup.
- Ensure that you have installed the recent versions of both Android Debug Bridge (adb) and Android Asset Packaging Tool (AAPT) and added the location of those tools to the system path of your machine. For information about installing those tools, see Setting up CTS: ADB and AAPT.
- Download the released CTS builds onto your Linux host machine and unzip the build to host location.
- Connect at least one device. To prepare the DUT:
- Press the home button to set the device to the home screen.
- Don't use the DUT for any other tasks.
- Keep the DUT in a stationary position to avoid triggering sensor activity.
- Point the device camera at an object that can be focused.
- Do not press any keys on the device while the CTS is running. Pressing keys or touching the screen of a DUT interferes with the running tests and may lead to test failures.
- Launch the CTS console cts-tradefed script from
the folder where the CTS package has been unzipped. On the host's
command-line shell, run:
./android-cts/tools/cts-tradefed
- Run the default test plan (contains all test packages):
cts-tradefed > run cts
- If you want to improve test execution time, you can
shard the tests across multiple devices. Sharding requires the
host to connect at least two devices, but six or more devices are
recommended for efficiency. When sharding more than 1 device:
- For Android 9 and higher, use the command option
--shard-count number_of_shards
- For Android 8.1 and lower, use the command option
--shards number_of_shards
- For Android 9 and higher, use the command option
- If you don't want to run the whole test suite, you can run the
CTS plan of your choice from the
command line:
run cts --plan test_plan_name
To find the test plan name:
- For Android 7.0 and higher, to see a list of test modules, enter
list modules
- For Android 6.0 and lower, to view a list of test plans in the
repository, enter
list plans
- For Android 6.0 and lower, to view a list of test packages in the
repository, enter
list packages
- For Android 7.0 and higher, to see a list of test modules, enter
- For additional command options depending on CTS versions, refer to the console command reference, or under "help all" in the Tradefed console.
- If you want to improve test execution time, you can
shard the tests across multiple devices. Sharding requires the
host to connect at least two devices, but six or more devices are
recommended for efficiency. When sharding more than 1 device:
- Run multiple Retry sessions until all test modules are completed and the
test failure numbers are the same in the last two retry sessions.
- For Android 9 and higher, use
run retry --retry session_number --shard-count number_of_shards
- For Android 7.0–8.1, use
run cts --retry session_number --shards number_of_shards
- For additional Retry command options depending on CTS version, see CTS v2 Command Console.
- To understand implementation details for CTS retry, see Trade Federation Suite Retry.
- For Android 9 and higher, use
- View test progress and results reported on the console.