The scripts in this folder comprise a cross-platform unit test runner. The components of this runner are as follows:
1.) test_runner.py
This script is used to run unit test binaries on multiple platforms. Currently, it is only compatible with Linux and Android. To see a full list of parameters that can be supplied to the script, run “python test_runner.py --help”.
Running Tests:
To run all tests for a given platform, execute the “test_runner.py” script and provide at minimum the “--platform” and “--config” arguments. Example:
python test_runner.py --platform=android-x86 --config=devel
Running this command will run all unit tests for the “devel” build of the “android-x86” platform. To specify a device to run the tests on, provide the “--device_id” argument, as shown below:
python test_runner.py --platform=android-x86 --config=devel
--device_id=emulator-4
Running a Single Test:
If you would like to run a single unit test binary and view its output, you can do so by using the “--target_name” parameter and providing a test binary name, as shown below:
python test_runner.py --platform=android-x86 --config=devel
--device_id=emulator-4 --target_name=audio_test
Building Tests:
You can also use this script to build your unit test binaries before running them. To do this, provide the “-b” command line flag. If you would like to build the tests and then run them, provide the flags “-br”.
2.) Master list of test binaries
In your application's “starboard_configuration.py” file, define a variable called TEST_TARGETS. It should be a list containing the names of all of the test binaries that you want the test runner to run. An example is shown below:
TEST_TARGETS =[ “audio_test”, “bindings_test” ]
If your “starboard_configuration.py” file contained this list, then every platform you support in Starboard would try to run these test binaries unless they were filtered out as described below.
To filter out tests that you do not want to run for a specific platform, implement a method within the platform's GYP PlatformConfig class called “GetTestFilters()”. The PlatformConfig class lives in a gyp_configuration.py file. See “../../linux/x64x11/shared/gyp_configuration.py” for an example.
The “GetTestFilters()” function should return a list of TestFilter objects, which can be constructed by importing “./test_filter.py”. To make a test_filter object, provide the constructor with the test target name, the name of the actual unit test that the target runs, and optionally, the build configuration from which the test should be excluded. An example is shown below:
test_filter.TestFilter(‘math_test’, ‘Vector3dTest.IsZero’, ‘debug’)
If a configuration is not provided, then the test will be exluded from all configurations.
To filter out all tests for a particular target, provide “test_filter.FILTER_ALL” as the test name.
To disable unit testing for all targets and all configurations, return a list containing “test_filter.DISABLE_TESTING”.