Template Contract
A Sandswap template is a rootfs tree that the platform materializes into SandFS volumes (flat) or mounts as read-only OverlayFS lowerdirs (overlay). Template content is served to guests by the virtio-fs passthrough daemon (the sandvmm identity) and mounted by the exec sandlot identity — neither is the tree’s owner. Every read therefore travels through the shared group (the same grant the storage contract uses). The contract below is what makes that possible; it is implemented once, in sandlot/provision/template.NormalizeTemplateTree / ValidateTemplateTree, exposed as the output/template-tool CLI.
1. The rules
| # | Rule | Why |
|---|---|---|
| 1 | Directories: g+rx (group traverse + list) | the daemon resolves paths through them; without group traverse every guest read EACCESes |
| 2 | Regular files: g+r where the owner can read; g+x where the owner can execute | the daemon performs the guest’s reads as its own uid; the group bit is the grant |
| 3 | setuid/setgid/sticky are preserved as authored | inert host-side (nosuid,nodev mounts, storage contract row 9); guest-side semantics (su, umount) stay intact |
| 4 | Ownership inside the tree is irrelevant | materialization ignores tar uid/gid; the setgid store assigns the shared group |
| 5 | Symlinks: tree-internal relative targets (absolute targets are rewritten at extraction) | extraction containment |
| 6 | Device/FIFO/socket entries are not supported | inert under nodev; no template needs them |
| 7 | Content that must stay private (SSH private keys, credentials) belongs in home/ or workspace/, never the read-only rootfs | rootfs content is necessarily group-readable by the service identities |
| 8 | Group and other write bits are cleared (the owner keeps write; sticky/setuid/setgid survive) | template trees are shared storage; a tenant-chmod’ed 0777 entry would let another local identity rewrite content every tenant executes. Flat materialization re-adds group write (the agent’s shared-group path) but never other-write. |
2. Producing a conformant template
# 1. Build the tree however you like (debootstrap, tar extraction, cp -a …).
# 2. Normalize it with the canonical tool:
output/template-tool normalize /path/to/rootfs
# 3. Validate (also a build gate): the rules above PLUS the rootfs
# well-formedness rules in §3 — an off-contract template reads as EACCES
# inside every guest, one without /bin/sh never execs at all:
output/template-tool validate /path/to/rootfs
- The stock build scripts (
build/build-rootfs-template.sh,build/build-ubuntu-rootfs.sh) run both steps automatically; the deploy scripts that stage templates into the flat scan dir do the same andchgrpthe tree to the shared group (the group is the grant — do not “fix” readability withchmod -R a+rX, which widens to every local user instead of the service group). save-templateand OCI template pulls are normalized by the prov role itself at materialization; when it has to repair a non-conformant artifact it logs a warning naming the layer — runtemplate-tool normalizeon the source to fix it upstream.- Flat-template registration (
flat-templates/scan) validates and warns (it does not refuse: flat seeding repairs modes at consume, so foreign content cannot brick startup).
3. Rootfs well-formedness: template-tool validate
validate is the full offline conformance pass over a template tree — the modes contract of §1 plus everything a runtime consumer needs from the tree’s SHAPE. It mirrors checks the platform applies at consume time (warden exec, mount-time validation, import paths), so a template that passes cannot fail a runtime check for a STATIC reason:
| # | Rule | Severity | Why / runtime counterpart |
|---|---|---|---|
| 1 | the §1 modes contract (same findings as check) | error | the guest read path (§1) |
| 2 | /bin/sh resolves, via symlinks with chroot semantics (absolute targets resolve against the tree root), to an executable regular file | error | the warden execs /bin/sh -c for every agent command; a root without it cannot exec |
| 3 | no volume control files at the tree root (.sandfs-id, .provisioning, .save-staging) | error | volume state is never template content; save-as-template strips them — a template carrying them leaks the source volume’s metadata into every guest |
| 4 | the attach-critical skeleton entries dev proc workspace home are real directories | missing, symlinked or not a directory → error | the warden secures each with a FATAL mount step (open_dir: mkdirat on ENOENT, refuse a symlink); a read-only overlay root cannot be repaired, so the whole attach is refused — and the import paths cannot create or replace the entry either |
| 5 | the soft skeleton entries bin etc resolve to a directory (same chroot semantics as /bin/sh; a usr-merge bin -> usr/bin is legitimate) | missing → warning; dangling symlink, unresolvable or not a directory → error | these degrade instead of refusing (the busybox hot-bind is skipped, resolv.conf injection is best-effort) — but a dangling link is still unrepairable by the import paths |
| 6 | sys/ and tmp/ exist (same chroot-semantic resolution) | warning | the warden mounts the real proc/dev inside the guest and expects them (empty is fine) |
| 7 | no device/FIFO/socket entries | warning | unsupported (§1 rule 6); skipped when the template is pushed — a saved layer may carry tenant-planted ones, a hand-built tree should not |
Errors fail the gate (exit 1); warnings and info print but exit 0. Given a scan root (<name>/<version>/ nesting, or a bare <name>/ rootfs) the tool descends and validates every leaf tree — the same classification the registration scans apply. Given a store directory (refs/ or layers/) it refuses and points at check-store: layer trees under layers/ are fragments, not complete rootfs trees.
4. Store integrity: template-tool check-store
output/template-tool check-store /var/sandswap/templates \
--flat-root /var/sandswap/flat-templates
The offline pass over a template store (refs/ + layers/). Every rule mirrors a check the two runtime ref loaders and the layer validators enforce at consume time — on shared storage a peer node or a tampered store can rewrite a ref, which is exactly why those checks exist; this finds the same defects before a template reaches a consumer:
| # | Rule | Severity | Runtime counterpart |
|---|---|---|---|
| 1 | every refs/*.json is a regular file (bounded size) that parses; the entry’s name/version encode to the ref file’s base name | non-regular or oversized → error; unparseable → error; identity mismatch → error (the :↔_ filename encoding is not injective — the cross-tenant rootfs-substitution class; every loader refuses it); legacy or partial identity → warning | the ref loaders; a FIFO or device at a ref name would block a reader forever, and no writer produces one |
| 2 | mode is flat/overlay/empty and agrees with the filename suffix (a recorded empty mode reads as overlay) | all disagreements → warning (readers select the file by filename suffix and never consult the field); a legacy no-suffix filename → warning (listing-only: loaders open only <ref>.<mode>.json) | the ref loaders |
| 3 | layer ids carry no :/, separator; relative ids stay under layers/ lexically; absolute overlay ids stay inside the store | error | the mount-time layer trust boundary (the :/, rule is validateLayerID’s overwrite of the lowerdir= option) |
| 4 | a layer’s RESOLVED path stays inside its per-consumer root: overlay → the store; flat → the store or a configured scan root | outside the store (overlay) / outside every root (flat) → error; a relative layer that leaves layers/ but stays in the store → warning (mounting and seeding accept it; packing refuses it) | the mount-time symlink leg (overlay.go) and the flat seeding boundary (--flat-root repeats the prov instance’s -flat-templates roots) |
| 5 | absolute flat layer paths are lexically inside the store or a configured scan root | unjudgeable (no --flat-root given) → warning; outside every given root → error | the flat seeding trust boundary — pass the PROV instance’s -flat-templates path as the refs record it (a shared store records the PROV host’s mount path, which differs from other hosts’ view) |
| 6 | every named layer exists and is a directory | store-owned relative layer → error; absolute scan-root source → warning (durable ref by design: re-creating the source directory restores the template) | read-time existence (a missing layer hides the ref from listings and fails creation fast) |
| 7 | recorded OCI layer digests parse; an oci_layers-only ref is stored as flat; a ref does not mix layers with oci_layers | unparseable digest or an OCI-only overlay ref → error; mixed provenance → warning | only the flat creation path consumes oci_layers; the overlay mount loader refuses a ref with no local layers |
| 8 | layers/ naming (sha256-<digest> / local-<name>-<version>), orphaned layers, writer debris (staging dirs, publish markers, save locks) | warning / info | reachability-GC inverse; the staging reaper |
5. What the contract does not require
- No specific owner/uid (rule 4).
- No world access. Templates are readable by exactly: root, the file’s owner, and the shared group — nothing else on the host.
- No size or layout constraints (beyond the platform’s general path limits).