Pantavisor needs store some artifacts on disk. These are the elements on the root of the /storage space:
- boot: information written by Pantavisor so bootloader can run the proper revision after boot up.
- cache: to store metadata and other stuff that is not fixed to revisions.
- config: to store part of the configuration.
- disks: permanent and revision disks.
- logs: both containers and Pantavisor logs.
- objects: binary artifacts that are part of revisions.
- trails: list of revisions.
There are currently 4 disks types supported for disk:
- Non-encrypted directory
- Device Mapper crypt using without hardware acceleration (versatile)
- Device Mapper crypt using i.Mx CAAM
- Device Mapper crypt using i.Mx DCP
The new disk v2 supports all the previous ones and adds 2 new types:
- Swap disk based on zram
- Volume disk based on zram
Additional volumes can be specified to be used internally by Pantavisor. There are two volumes that are currently supported:
- pv--devmeta: if defined, Pantavisor will use this volume to save and load device metadata.
- pv--usrmeta: if defined, Pantavisor will use this volume to save and load user metadata.
Metadata is just a list of key-value pairs of string type. This list is persistent and storaged on disk. By default, it is saved in plain text, but this can be changed in favour of volumes on a crypt disk.
This type of metadata is meant to be generated by Pantavisor or any of the running containers to expose internal information.
Check the list of default key-value pairs that Pantavisor sends.
This type of metadata is meant to be generated by the user so Pantavisor or any of the containers can consume it.
Check out the list of default user metadata consumed by Pantavisor.
Pantavisor can centralize all of your container logs, separated by revision, in one place on-disk. It does so by running a small server (Log Server) which offers a couple of sockets to the containers:
There are a number of parameters that can be tweaked from the configuration that will affect Log Server, such as
log.level, etc. Between those, there is an important parameter called
log.server.outputs that allows to change how the logs will be stored:
These options are not mutually exclusive and can be combined in any fashion.
This is the default option. In this case the log output will be delivered in different files by containers. Pantavisor logs will also be stored in its own directory.
With this option, all logs will be centralized in one file. These logs are delivered in a JSON following this structure:
"msg": "starting logserver loop"
Outputs logs into stdout, which can be useful for debugging.
Standard Output Pantavisor¶
Same as Standard Ouput but filtering all messages except Pantavisor ones.
Standard Output Containers¶
Same as Standard Ouput but filtering all messages except container ones.
Send logs to /dev/null.
All the log outputs, except for the NULL Sink, will print the timestamp along the log lines. It is possible to configure the format of the timestamp with the configuration key
log.<output>.timestamp.format, which can be set with the following values:
|"01/02 03:04:05PM '06 -0700"
|"Mon Jan 02 15:04:05 -0700 2006"
|"Mon Jan _2 15:04:05 2006"
|"02 Jan 06 15:04 -0700"
|"Mon, 02 Jan 2006 15:04:05 -0700"
Those formats are based in the Golang time constants. For other formats, the strftime formatters can be used setting the
strftime: prefix. For example:
strftime:%d, %T %Y. Please check the strftime manual for more information about formatters.
Trails and objects¶
Pantavisor trails, with their state JSONs are stored for each installed revision. The rest of artifacts that can be shared between revisions, are called objects and stored separately.
The Pantavisor garbage collector is in charge of cleaning up the /storage by automatically removing unused Pantavisor artifacts.
The garbage collector will not affect any of the files related to the running revision, the revision that is being updated or the latest DONE revision. These revisions that are not affected by the garbage collector can be expanded to the factory revision using the storage.gc.keep_factory configuration parameter.
Currently, it can be triggered by three different events:
- by a remote update that requires more disk space than available
- if a threshold of disk usage is surpassed
- with a command
This type of trigger will not work with local updates. For those, you will need to use one of the following two options.
Disk usage will be checked periodically and garbage collector will be activated if that threshold is reached. By default it is disabled. This can be changed with the storage.gc.threshold parameter.
If an object is put using the objects endpoint, the threshold will be disabled temporarily, which is done to avoid removing objects that could be linked to the upcoming new revision. This parameter can be changed from the configuration.
Finally, our garbage collector can be triggered on demand issuing a command through Pantavisor control socket.
Pantavisor offers secureboot, a security mechanism to ensure integrity of the artifacts that are part of a revision. As revisions are composed of a state JSON and a series of artifacts pointed by that JSON, Pantavisor can perform a double level validation: artifact checksum and state signature.
Objects and JSONs belonging to the revision that is about to be run after booting up are checked against the checksums stored in their state JSON by default.
As running a full SHA hashsum of all artifacts can be time consuming for low spec targets and thus increas boot up time, a lazy integrity validation on container storage can be performed using handlers, which are specified at the state JSON. Handlers are scripts to mount/unmount volumes. The only supported handler so far is dm-verity.
Checksum can also be totally disabled at the configuration level if boot up speed needs to be boosted further.
Additionally to artifact checksum, the state JSON can be signed. This can be done from your host computer or from an automated CI, and it is then validated from Pantavisor using a cryptographic hash algorithm. It can be tuned according to one of these three levels of severity:
- disabled: no signature validation will be performed.
- audit: all validations from lenient and strict will be performed, errors will be reflected in log but nothing will fail.
- lenient: only the signed artifacts in the revision will be validated. Enabled by default.
- strict: all artifacts in the revision must be signed and will be validated.
Right now, RS256, ES256, ES384 and ES512 algorithms are supported and Pantavisor must know the public key either by storing it on disk directly or by using a certificate chain.
Validation is performed in Pantavisor in two places:
- when booting up, right after Pantavisor is initialized and before the current revision is loaded. In case of failure, the update will be reported as ERROR if possible and the device will either rollback or just reset.
- when a new remote update is received or a new local update is run. In case of failure, the update will be reported as WONTGO.
On-disk public key¶
In this option, Pantavisor will need the RSA public key to be stored on disk. To do so, it is necessary to add it during compiling time with the PV_PVS_PUB_PEM option.
It will need the signature and information about how to generate it in the revision checkout.
Certificate chain of trust¶
In this case, the public key will travel in the revision checkout in a x509 certificate. The rest of the chain can be added to the checkout, but the root certificate have to be stored on disk during compiling time using the PV_PVS_CERT_PEM option.
It will also need the signature and information about how to generate it in the revision checkout.