Instrumentation tests

First read Test your app on developer.android.com. Take note there are some differences in how instrumentation tests are used in platform testing.

In summary, an instrumentation test provides a special test execution environment as launched via the am instrument command, where the targeted application process is restarted and initialized with basic application context, and an instrumentation thread is started inside the application process VM. Your test code starts execution on this instrumentation thread and is provided with an Instrumentation instance that provides access to the application context and APIs to manipulate the application process under test.

Key concepts

• an instrumentation must be declared in an application package, with an <instrumentation> tag nested under the <manifest> tag of the application package manifest.
• an application package manifest may technically contain multiple <instrumentation> tags, though it's not commonly used in this fashion.
• each <instrumentation> must contain:
  • an android:name attribute: it should be the name of a subclass of Instrumentation that's included in the test application, which is typically the test runner that's being used, e.g.: android.support.test.runner.AndroidJUnitRunner
  • an android:targetPackage attribute must be defined. Its value should be set to the application package under test.

Summary of steps

1. Below are common destinations for hermetic tests against framework services:

   frameworks/base/core/tests/coretests
   frameworks/base/services/tests/servicestests
   

   If you are adding a brand new instrumentation module for your component, see

   • Self-Instrumenting Tests: A Complete Example
   • Instrumentation Targeting an Application: A Complete Example

2. Following the existing convention if you are adding tests into one of the locations above. If you are setting up a new test module, please follow the setup of AndroidManifest.xml and Android.mk in one of the locations above

3. See frameworks/base/core/tests/coretests/ for an example. Note these lines install extra apps:

   <option name="test-file-name" value="FrameworksCoreTests.apk" />
   <option name="test-file-name" value="BstatsTestApp.apk" />
   

4. Do not forget to mark your test as @SmallTest, @MediumTest or @LargeTest

5. Build the test module with m, e.g.:

   m FrameworksCoreTests
   

6. Run the tests:

   • The simplest solution is to use Atest like so:

     atest FrameworksCoreTests
     

   • Or for more complex tests, use the Trade Federation test Harness:

   m tradefed-all
   tradefed.sh run template/local_min --template:map test=FrameworksCoreTests
   

7. If not using Tradefed, manually install and run the tests:

   1. Install the generated apk:

   adb install -r ${OUT}/data/app/FrameworksCoreTests/FrameworksCoreTests.apk
   

   Tip: you use adb shell pm list instrumentation to find the instrumentations inside the apk just installed

   1. Run the tests with various options:

      1. all tests in the apk

         adb shell am instrument -w com.android.frameworks.coretests\
           /android.support.test.runner.AndroidJUnitRunner
         

      2. all tests under a specific Java package

         adb shell am instrument -w -e package android.animation \
           com.android.frameworks.coretests\
           /android.support.test.runner.AndroidJUnitRunner
         

      3. all tests under a specific class

         adb shell am instrument -w -e class \
           android.animation.AnimatorSetEventsTest \
           com.android.frameworks.coretests\
           /android.support.test.runner.AndroidJUnitRunner
         

      4. a specific test method

         adb shell am instrument -w -e class \
           android.animation.AnimatorSetEventsTest#testCancel \
           com.android.frameworks.coretests\
           /android.support.test.runner.AndroidJUnitRunner
         

Your test can make an explicit assertion on pass or fail using JUnit APIs; in addition, any uncaught exceptions will also cause a functional failure.

To emit performance metrics, your test code can call Instrumentation#sendStatus to send out a list of key-value pairs. It's important to note that:

1. metrics can be integer or floating point
2. any non-numerical values will be discarded
3. your test apk can be either functional tests or metrics tests, however mixing both are not currently supported