Skip to content

Pantavisor Test Framework

The Pantavisor Test Framework contains a set of scripts to permit automated testing on Pantavisor taking advantage of the App Engine init mode.

Requirements

Firstly, to be able to run ARM containers on a x64 host in an apt based system:

sudo apt install binfmt-support docker.io git jq qemu qemu-user-static
sudo update-binfmts --enable qemu-arm

Now, you need to get the Pantavisor BSP source code.

repo init -u https://gitlab.com/pantacor/pv-manifest -m release.xml
repo sync -j10 -c --no-clone-bundle

If you prefer to sync by group with the -g option, remember to add the x64-appengine target:

repo init -u https://gitlab.com/pantacor/pv-manifest -m release.xml -g runtime,x64-appengine
repo sync -j10 -c --no-clone-bundle

How to Run All Tests

Now, you have to build Pantavisor for the x64-appengine target:

PANTAVISOR_DEBUG=yes ./build.docker.sh x64-appengine

The resulting tarball will be used by the test script. To run all tests with it:

./test.docker.sh run

This might take a while, but you can always stop it with CTRL-C.

How to Run a Test

Tests can be individually run too.

First, it is important to know which tests are available:

./test.docker.sh ls

For example, to run the test number 0 from the 'local' group of tests:

./test.docker.sh run local:0

To run all tests from the local group of tests:

./test.docker.sh run local

How to Create a New Test

To create a new test inside of the 'local' test group:

./test.docker.sh add local

This will create the necessary templates for a new test in tests/local/data/roles. These templates can then be edited manually to get the desired test.

How to Modify the behavior of an Existing Test

If we go to any newly created test using the previous command, we will find a directory and a JSON file:

output  resources  test.json

The test.json file contains the necessary metadata information for the execution and evaluation of a test:

{
        "#spec": "pv-test@1",
        "description": "ls query to mgmt and non-mgmt containers",
        "setup": {
                "cmdline": "",
                "pantavisor.config": "../../common/configs/pantavisor.config",
                "pvs": "../../common/pvs/pvs.defaultkeys.tar.gz",
                "containers": {
                        "control": "pvr-sdk",
                        "tarballs": [
                        ],
                        "urls": [
                                "https://pvr.pantahub.com/pantahub-ci/pv_tests_assets/3#pvr-sdk/",
                                "https://pvr.pantahub.com/pantahub-ci/pv_tests_assets/3#pvr-sdk_norole/"
                        ]
                },
                "ready-script": "resources/ready",
        },
        "test-script": "resources/test",
        "skip": "false"
}

test.json

#spec

Version of the test.json. Only supported value is 'pv-test@1'.

description

Brief description of the test objective.

setup

Initial Pantavisor setup for the test.

cmdline

Substitutes what is read from /proc/cmdline. Can be used for configuraiton purposes.

pantavisor.config

Pantavisor configuration file that will be used for the test.

pvs

Tarball with the secureboot certs and keys. This makes possible to use tests from different manifests in our own.

containers

Initial container setup for the test.

control

The name of the container that will be used by the Test Framework utilities as the management controller for things such as initialization and shutdown.

It is important to set a control container for the correct operation of the test (unless what is being tested is a device without management containers), and it is important to notice that the container must be provided either in the list of tarballs or the list of urls.

tarballs

List of tarballs to be installed in the first revision for the test. Can be left empty.

urls

List of Clone URLs to be installed in the first revision for the test. Can be left empty.

ready-script

POSIX-compliant sh script to be executed before evaluating if Pantavisor status is READY. Can be used to force a READY status for any container.

test-script

The test itself. Must be a POSIX-compliant sh script. The stdout and stderr of it will form the result of the test.

skip

Whether to skip a test or not. Possible values are 'true' or 'false'.

The test will still be executable individually or interactively.

How to Debug a Test

To interactively run a test:

./test.docker.sh run local:0 -i

A console will be opened with an instance of App Engine already running with the parameters that were set in test.json, except the test script will not be automatically run.

If Pantavisor is not getting to the READY status, you can use the manual option plus the interactive one:

./test.docker.sh run local:0 -i -m

With this, the console will be opened right before starting Pantavisor. You will be able to start it with a modified configuration, such as enabling stdoud logs for debugging.

How to Modify the Result of a Test

To create or overwrite the results of a test:

./test.docker.sh run local:0 -o