CTS (Compatibility Test Suite) - AOSP Expert & Production Engineering

CTS Purpose: Android Compatibility Assurance

The Compatibility Test Suite (CTS) is the cornerstone of the Android ecosystem. It is an automated testing suite provided by Google to ensure that devices built by OEMs comply with the Android Compatibility Definition Document (CDD).

If a device passes CTS, it guarantees that third-party applications written against the public Android SDK/NDK will run correctly without unexpected crashes or missing APIs. Passing CTS is a hard requirement for licensing Google Mobile Services (GMS), including the Google Play Store.

Running CTS Locally

AOSP developers frequently run CTS modules locally to verify that framework modifications do not break existing public APIs. You can download pre-built CTS binaries from the Android open source site or build it directly from source.

To build and run a CTS module locally using atest:

# Build the CTS environment
source build/envsetup.sh
lunch aosp_husky-trunk_staging-userdebug

# Run a specific CTS module (e.g., standard permission tests)
atest CtsPermissionTestCases

Alternatively, you can run the full CTS console directly:

# Start the interactive CTS tradefed console
./android-cts/tools/cts-tradefed

# Inside the console, run a module
cts-tf > run cts -m CtsAppSecurityHostTestCases

CTS Test Categories

CTS consists of millions of individual tests covering every facet of the operating system. These are organized into modules, including:

API Tests: Ensure all public classes, methods, and constants defined in the SDK exist and behave exactly as documented.
Platform Security: Verify sandboxing, permission enforcement, and cryptographic implementations.
Hardware Diagnostics: Test camera HALs, audio latency, sensor accuracy, and graphics drivers.
Media Compliance: Validate audio/video codecs, ensuring standard formats decode correctly.

Investigating CTS Failures

When a CTS test fails, it indicates a violation of the CDD. The failure must be analyzed and resolved.

Retrieve Logs: Pull the XML result report and the device logcat from the TradeFed output directory.
Identify the Assertion: Locate the exact JUnit assertion that failed in the CTS source code (located in cts/tests/).
Trace the Framework Code: Determine how the framework implementation deviates from the expected behavior.
Fix or Waive: Usually, the AOSP code must be fixed. In extremely rare cases involving hardware limitations, OEMs may apply to Google for a waiver.

# Check device logs for CTS failures
adb logcat -d | grep -i "TestRunner"

CTS-in-a-box for OEM Testing

Running the entire CTS suite takes days and requires a specialized setup, including specific Wi-Fi access points, Bluetooth beacons, and IPv6 networks.

"CTS-in-a-box" refers to a standardized, automated hardware rig designed to execute the full suite without human intervention. OEMs maintain racks of these boxes to run continuous CTS regression testing on every nightly firmware build, ensuring that new driver updates or kernel patches do not inadvertently break Android compatibility.