Manufacturer guide for long-term Android security

This guide describes Google recommended best practices for applying security patches that are evaluated by the Android Compatibility Test Suite (CTS). It's intended for manufacturers of Android-compatible OEM equipment (manufacturers) that will be supported longer than three years such as vehicles, TVs, set-top boxes, and home appliances. This guide is not intended for end users (for example, vehicle owners).

Acknowledgements and disclaimers

This guide doesn't legally or contractually bind Google or other manufacturers and isn't intended to be a set of requirements. Rather, this guide is an instructional aid that describes recommended practices.

Feedback

This guide isn't intended to be comprehensive; additional revisions are planned. Submit feedback to manufacturers-guide-android@googlegroups.com.

Glossary

Term Definition
ACC Android Compatibility Commitment. Formerly known as the Android Anti-Fragmentation Agreement (AFA).
AOSP Android Open Source Project
ASB Android Security Bulletin
BSP Board Support Package
CDD Compatibility Definition Document
CTS Compatibility Test Suite
FOTA firmware over the air
GPS global positioning system
MISRA Motor Industry Software Reliability Association
NIST National Institute of Standards and Technology
OBD on-board diagnostics (OBD-II is an improvement over OBD-I in both capability and standardization)
OEM original equipment manufacturer
OS operating system
SEI Software Engineering Institute
SoC system on chip
SOP start of production
SPL Security Patch Level
TPMS tire-pressure monitoring system

About Android OS

Android is an open source, Linux-based full software stack designed for a variety of devices and form factors. Since its first release in 2008, Android has become the most popular Operating System (OS), powering 1.4+ billion devices worldwide (2016). Approximately 67% of those devices use Android 5.0 (Lollipop) or newer as of March 2017 (more recent figures are available on the Android Dashboard). While the vast majority of devices are mobile phones and tablets, Android is growing in smartwatches, TVs, and automotive in-vehicle infotainment (IVI) devices.

The number of Android apps available in Google Play Store has reached 2.2+ million (2016). Android app development is backed by the Android Compatibility program, which defines a set of requirements via Compatibility Definition Document (CDD) and provides testing tools via the Compatibility Test Suite (CTS). The Android compatibility programs ensures any Android app can run on any Android-compatible device that supports the required features for the app.

Google releases new OS versions, OS security updates, and information about discovered vulnerabilities on a regular basis. The Android Security Bulletin should be reviewed by manufacturers for the applicability of these updates to Android OS supported products. For a review of Android security, compatibility, and build systems, see the following:

About connected vehicles (canonical long-lived products)

Vehicles started being connected with the introduction of AM radio in the 1920s. From there, the number of external physical and wireless connections began to grow as regulators and automakers turned to electronics to ease diagnostics and service (for example, OBD-II port), improve safety (for example, TPMS), and meet fuel economy targets. The following wave of connectivity introduced driver convenience features such as remote keyless entry, telematics systems, and advanced infotainment features such as Bluetooth, Wi-Fi, and smartphone projection. Today, integrated sensors and connectivity (for example, GPS) support safety and semi-autonomous driving systems.

As the number of vehicle connections increase, so does the area of the potential vehicle attack surface. Connections bring with them a similar collection of cybersecurity concerns as for consumer electronics. However, while reboots, daily patch updates, and unexplained behaviors are the norm for consumer electronics, they are inconsistent for products with safety-critical systems such as vehicles.

Manufacturers must take a proactive approach to ensure the continued security and safety posture of a product in the field. In short, manufacturers must be aware of known security vulnerabilities in the product and take a risk-based approach to addressing them.

Ensure long-term security

A connected vehicle often has one or more electronic control units (ECUs) that include multiple software components such as OS, libraries, utilities, etc. Manufacturers should track such components and identify known published vulnerabilities with proactive analysis, including:

  • Regularly evaluating the product against Common Vulnerabilities and Exposures (CVE) database.
  • Intelligence gathering for product-related security flaws.
  • Security testing.
  • Actively analyzing Android Security Bulletins.

Example OS and security patch updates (IVIs running Android):

Figure 1. Sample major OS and security update rollout over vehicle lifetime.

# Step Activities

①

 Development Branch The manufacturer selects a version of Android (Android X). In this example, "Android X" becomes the basis of what will ship in the vehicle two years prior to initial Start of Production (SOP).
② Initial Launch Some months before Android X becomes the first OS version shipped in the product, security updates are taken from Android Security Bulletins (ASBs) and potentially other sources deemed valuable by the manufacturer. y2 = the second Security Bulletin for version X of Android, applied (backported) by the manufacturer to Android X. This update ships in the product and the production clock starts ticking at Year Zero with Android X.y2.

In this example, the manufacturer made the decision not to ship the more recent Android X+1 annual release. Reasons for shipping the most recent release include adding new features, addressing new security vulnerabilities, and/or shipping Google or third-party services that require the newer Android version. Reasons against shipping with the most recent release is the lack of time inherent to the vehicle development and launch process required to integrate, test, and validate the changes, including compliance with all regulatory and certification requirements.
③ Full OS Update Post-SOP, the manufacturer releases Android X+2 OS update, which is two Android releases after the version used for initial product (Android X0). ASB security updates are available for the API level (as of the ship date), so the update goes out as X+2.y0 approximately 1.25 years after SOP. This OS update may or may not be compatible with fielded products. If it is, a plan could be created to update deployed vehicles.

Unless other business agreements are in place, the decision to make a full OS update is entirely at the discretion of the manufacturer.
④ Security Update Two years into the production lifespan of the vehicle, the manufacturer patches the Android X+2 OS. This decision is based on the manufacturer's risk assessment. The manufacturer chooses the third ASB security update for release X+2 as the basis of the update. Products receiving the security update are now are at (X+2.y3) OS + Android Security Patch Level.

While manufacturers can select individual security patches from any individual ASB, they must fix all required issues in the bulletin to use the Android security patch level (SPL) associated with the bulletin (for example, 2017-02-05). It is the manufacturer's responsibility to perform the backport and security release for the supported product.
⑤ Full OS Update A repeat of step 3 (Full OS Update), the second full OS update brings the product up to Android X+4, three years into the production lifespan of the vehicle. The manufacturer is now balancing the newer hardware requirements of a recent Android version against the hardware in the product and the user benefits from an updated Android OS. The manufacturer releases an update without security updates, so the product is now at (X+4.y0) OS + Android Security Patch Level.

In this example, due to hardware limitations, X+4 is the last major version of Android that will be provided for this product, although 6+ years of expected lifetime for the vehicle still requires security support.
⑥ Security Update A repeat of step 4 (Security Update). The manufacturer has the task of taking ASB security updates from a much later version of Android (X+6) and porting some or all of those updates back to Android X+4. It is the manufacturer's responsibility to merge, integrate, and perform the updates (or contract with a third-party). Also, the manufacturer should be aware that security issues in versions of Android that are no longer supported aren't covered in ASB.
⑦ Security Update Eight years into the production lifecycle of the vehicle, four Android releases since the last OS update in Step 5 (Full OS Update), and ten years since Android X was specified, the burden of curating and backporting security patches is wholly on the manufacturer for those versions older than three years from the API level public release.

Security best practices

To make security compromises more difficult, Google recommends and employs the use of commonly accepted best practices for security and software engineering, as described in Implementing Security.

Security guidelines

Recommended practices for security include:

  • Use latest versions of external libraries and open source components.
  • Don't include intrusive debug functionality in release versions of the OS.
  • Remove unused functionality (to reduce unneeded attack surface).
  • Use the least privilege principle and other Android app development best practices.

Software development guidelines

Recommended practices for secure software development for the lifecycle of the system include:

  • Perform threat modeling to rank and identify assets, threats, and potential mitigations.
  • Perform architecture/design review to ensure secure and sound design.
  • Perform regular code reviews to identify anti-patterns and bugs as soon as possible.
  • Design, implement, and run high-code coverage unit tests, including:
    • Functional testing (including negative test cases)
    • Regular regression testing (to ensure fixed bugs don't resurface)
    • Fuzz testing (as part of the unit test suite)
  • Use static source code analysis tools (scan-build, lint, etc.) to identify potential problems.
  • Use dynamic source code analysis tools, such as AddressSanitizer, UndefinedBehaviorSanitizer, and FORTIFY_SOURCE (for native components) to identify and mitigate potential problems during system development.
  • Have a management strategy for software source code and release configuration/version.
  • Have a patch management strategy for generation and deployment of software patches.

Security backport policy

Google currently provides active support for security backports of discovered and reported security vulnerabilities for three (3) years from the API level public release. Active support consists of the following:

  1. Receive and investigate vulnerability reports.
  2. Create, test, and release security updates.
  3. Provide recurring releases of security updates and security bulletin details.
  4. Perform severity assessment as per established guidelines.

After three years from the API level public release date, Google recommends the following guidelines:

  • Use a third-party (such as SoC vendor or Kernel provider) for backport support for OS security updates older than three years from API release.
  • Use a third-party to perform code reviews using the publicly provided ASBs. While ASBs identify vulnerabilities for currently supported version, a manufacturer may use the provided information to compare the newly released updates against prior versions. This data can be used to perform impact analysis and potentially generate similar patches for OS versions older than three years from API release.
  • When appropriate, upload security updates to the Android Open Source Project (AOSP).
  • The manufacturer must coordinate the handling of security updates for vendor-specific code (for example, proprietary device-specific code).
  • The manufacturer should join the NDA Android Security Bulletin Partner Preview notification group (requires the signing of legal agreements such as the Developer NDA). Bulletins should include:
    • Announcements
    • Summary of issues by patch level, including CVE and severity
    • Vulnerability details where appropriate

Additional references

For instructions on secure coding and software development practices, refer to the following:

Google encourages the use of the following recommended practices.

It is generally recommended that any connected product be launched with the latest OS version, and a manufacturer should attempt to use the most recent OS version before launching the product. While locking down the version is necessary to drive stability prior to testing and validation, the manufacturer must balance the product stability gained from older OS versions with newer OS versions that have fewer known security vulnerabilities and enhanced security protections.

Recommended guidelines include:

  • Due to long development lead times inherent to the vehicle development process, manufacturers may need to launch with OS version n-2 or older.
  • Maintain conformance with Android Compatibility for each released Android OS version with an over-the-air (OTA) campaign.
  • Implement product Android Firmware-over-the-air (FOTA)-capable for fast, customer-friendly updates. FOTA should be done using security best practices such as code signing and TLS connection between product and IT backoffice.
  • Submit independently identified Android security vulnerabilities to the Android Security team.

Note: Google has contemplated device type or industry-specific notifications in Android Security Bulletins. However, because Google doesn't know the kernel, drivers, or chipsets for a given device (vehicle, TV, wearable, phone, etc.), Google doesn't have a deterministic way to label any given security issue with a device type.

The manufacturer should make every attempt to use latest OS version or security updates for the version in use during product cycle enhancements. Updates can be performed during recurring periodic product updates, or for hotfixes to address quality and/or other issues. Recommended practices include:

  • Create a plan to address driver, kernel, and protocol updates.
  • Utilize an industry appropriate method for providing updates to deployed vehicles.

Compatibility Definition Document (CDD)

The Compatibility Definition Document (CDD) describes requirements for a device to be considered Android-compatible. The CDD is public and available to everyone; you can download CDD versions from Android 1.6 to the latest version from source.android.com.

Satisfying these requirements for a product involves the following basic steps:

  1. Partner signs the Android Compatibility Commitment (ACC) with Google. A Technical Solution Consultant (TSC) is then assigned as a guide.
  2. Partner completes the CDD review for the product's Android OS version.
  3. Partner runs and submits the CTS results (described below) until the results are acceptable for Android Compatibility.

Compatibility Test Suite (CTS)

The Compatibility Test Suite (CTS) testing tool verifies a product implementation is Android-compatible and that the latest security patches are included. CTS is public, open source, and available to everyone; you can download CTS versions from Android 1.6 to the latest version from source.android.com.

Each build of Android software released to the public (factory-install and field-update images) must prove Android Compatibility through CTS results. For example, if the device runs Android 7.1, the latest corresponding version of CDD 7.1 and CTS 7.1 should be referenced against when a release-intent build image is created and tested. Manufacturers are strongly encouraged to use CTS early and frequently to identify and remediate issues.

NOTE: Partners that sign other agreements, such as Google Mobile Services (GMS), may need to meet other requirements.

CTS workflow

The CTS workflow involves setting up the testing environment, running tests, interpreting results, and understanding the CTS source code. The following guidelines are intended to help CTS users (for example, developers, manufacturers) use the CTS effectively and efficiently.

  • Run tests frequently. CTS is designed as an automated tool that integrates into your build system. Running CTS frequently can help you find defects quickly and early when software degradation or regressions occur.
  • Download and examine the CTS source code. The full CTS source code is open source software that anyone can download and use (downloaded source code is fully buildable and runnable). When a test fails on the device, examining the relevant section of the source code can help you identify why.
  • Get the latest CTS. New Android releases can update the CTS with bug fixes, improvements, and new tests. Check CTS Downloads frequently and update your CTS program as necessary. Manufacturer and Google shall agree on the CTS version to pass for product launch as the product must be frozen at some point while the CTS continues to be refreshed.

Pass the CTS

For an Android Compatible product, Google ensures the device's CTS and the CTS Verifier reports test results are acceptable. In principle, all tests must pass. However, a test that fails for reasons other than the device not being in compliance with the Android Compatibility requirements is subject to review by Google. During this process:

  1. The manufacturer provides Google with the proposed CTS patches, patch validations, and justifications to prove the argument.
  2. Google examines the submitted material, and if accepted, updates the relevant CTS tests so that the device would pass in the next revision of CTS.

If a CTS test is suddenly failing after applying a security patch, the manufacturer must modify the patch so that is doesn't break compatibility OR show that test is wrong and provide a fix for the test (as described above).

The CTS remains open for reviews of test fixes. For example, Android 4.4 continues to accept fixes (see https://android-review.googlesource.com/#/c/273371/).

Frequently asked questions (FAQs)

Q: Who is responsible for applying security updates to a specific implementation of Android?

A: The manufacturer directly providing the device is responsible. This entity is not Google, which publishes security updates in AOSP and not for a specific device (such as a vehicle).

Q: How does Google handle security issues in Android?

A: Google continually investigates issues and developing potential fixes, which Google makes available to all supported API levels as part of regular security update process. Since August of 2015, Google has maintained a regular cadence of publishing bulletins and links to updates to source.android.com; Google also publishes security updates as part of major OS releases. See also the Security backport policy.

Q: If a manufacturer integrated all AOSP patches from an ASB but didn't integrate patches from BSP vendor mentioned in the same bulletin, can it still raise the security level (for example, apply the corresponding patch to platform/build)?

A: To declare an Android security patch level (SPL), a manufacturer must address all required issues published in the Android Security Bulletin (including prior bulletins) and mapped to a particular Android SPL. For example, a manufacturer using the March 2017 Security Bulletin (2017-03-01 SPL) has addressed all required issues documented in the March 2017 bulletin for that SPL and all prior updates including device-specific updates for all prior Android Security Bulletins, including the device-specific updates associated to the 2017-02-05 SPL.

Q: What happens when the manufacturer doesn't agree with security updates provided by BSP vendor OR when security updates mandated by an ASB aren't provided by vendors?

A: An ASB describes security vulnerabilities (enumerated by a list of CVEs) and often provides matching security tests. The goal is to ensure the listed vulnerabilities can no longer be reproduced on a device and that the device can pass associated security tests. As such, the issue is not about taking a security update provided by Google or a third-party vendor, but about the manufacturer attesting that the device isn't vulnerable to the list of CVEs in the ASB. The manufacturer is free to use the provided security updates or, if they have a change that is more appropriate to their device, use that change instead.

For example, consider a case in which Google addresses an AOSP security vulnerability using a code change that allows the component to remain fully functional and compliant with the CDD. If the manufacturer determines the component isn't needed on the device or isn't mandated by the CDD (or related certification testing), the manufacturer can remove the component to reduce future servicing needs and reduce attack surface. Although the manufacturer didn't use the provided security update, it did ensure the device isn't vulnerable to the CVE documented in the security bulletin. However, in deviating from the recommended security update, the manufacturer is taking a risk of incorrectly addressing the issue, introducing new security vulnerabilities, or otherwise reducing the functionality of the final build.

While we work with all SoC partners to ensure fixes exist for all issues in an ASB, we recommend that manufacturers secure a servicing agreement with their SoC vendors for the lifecycle of a device. SoCs might stop servicing a chipset earlier than desired, so establishing agreements prior to device chipset selection is an important part of the device launch process.

Finally, in cases where it's impossible to directly acquire or independently create a fix for an issue documented in an ASB, a manufacturer could maintain the prior Android SPL and still add the new available fixes to the build. However, this practice will eventually lead to issues with build certification (as Android ensures the latest security patch level is available on certified devices). Google recommens working with your SoC in advance to avoid this practice.

Q: If the manufacturer determines an ASB item isn't applicable for their product, does the item still need to be applied or patched in order to meet the other Google requirements or to pass CTS?

A: We don't require patches to be taken in order to declare an Android security patch level (SPL); we do require that the manufacturer attests that their build isn't vulnerable to the issue.

One example is where a component being patched doesn't exist in the manufacturer's system, or a component is removed from the manufacturer's system to address an issue. In that case the system may be compliant without requiring the manufacturer to take a patch.

This is fundamentally different from a manufacturer wanting, for example, to only fix critical patches, while not taking other applicable patches which would cause a security test to fail. In this case the SPL is assumed not to have been met.