Vendor Test Suite (VTS) and infrastructure

The Android Vendor Test Suite (VTS) provides extensive testing on the following:

  • Kernel
  • Hardware abstraction layer (HAL)

VTS runs on a desktop machine and executes test cases directly on attached devices or on the emulators. Similar to CTS, VTS is an automated test suite that uses the following major software components.

  • The VTS Trade Federation test harness runs on your host machine and manages test execution. It offers the ability to Configure Sharding across multiple devices under test (DUTs). You can also use the Suite Retry feature to retry only the failures rather than the entire test suites, which greatly reduces the re-run time.
  • Individual test cases are executed on the DUT. The test cases can be gtest-style tests, kernel tests, or JUnit-style tests written in Java.

Types of tests

The various types of VTS tests are described in the following sections.

gtest-style tests

Most tests in VTS are gtest-style tests that check the HAL implementation. The test is written in C++ and runs on the device. A typical VTS gtest iterates through each instance of a given interface, and runs all the test cases against it. Refer to VtsHalHealthStorageV1_0TargetTest for an example.

Linux kernel tests

  • Kselftest (external/linux-kselftest) is a collection of tests included within the Linux kernel repository at tools/testing/selftests, of which 23 are included in VTS to run on ARM.

  • Linux Test Project (external/ltp) tests validate the reliability, robustness, and stability of the Linux kernel.

JUnit-style tests

A small set of host-driven tests in VTS are JUnit-style tests, for example, KernelApiSysfsTest. The Java tests are implemented as BaseHostJUnit4Test, which is associated with a test device and can run shell commands to perform validation.

Standalone Python3 tests

Some VTS tests, such as vts_treble_sys_prop_test are written in Python3. The Python-based tests are implemented as unittest.TestCase and each test case can interact with the device through shell commands.