vsockguard — BPF LSM vsock access control
Scope. This page covers active impersonation only — a local user
connect()ing to a warden and sending mount/exec frames. The complementary threat, passive sniffing of an existing sandlot↔warden vsock connection, is not vsockguard’s job; it is prevented by the host security invariants in Host Prerequisites (and verified bysecurity/verify-host-security.sh). Both layers must hold.
1. Problem
Every Sandswap microVM’s warden listens on vsock port 1024. Its per-frame authentication (a 32-byte per-VM session key, delivered over the first host connection; every request carries a tag verified before the body is read) is what actually stops a caller who has not been handed the key. vsockguard is defense in depth on top of that: it keeps every other uid from reaching the port at all, so unauthenticated probes never reach warden parsing code. A non-privileged local user on the sandlot host can otherwise reach any warden with a few lines of code:
int fd = socket(AF_VSOCK, SOCK_STREAM, 0); // any UID, no device node
struct sockaddr_vm sa = { .svm_family = AF_VSOCK,
.svm_cid = 3, // guest CID (guessable: 3,4,5…)
.svm_port = 1024 };
connect(fd, (struct sockaddr *)&sa, sizeof(sa)); // straight into the warden
// then send mount_workspace + exec frames → read/modify/delete agent data
The root cause: AF_VSOCK bypasses the device nodes entirely. Opening the socket needs no device node — socket(AF_VSOCK, …) succeeds for any uid, and the enforcement point is connect(). /dev/vhost-vsock (0600, the sandvmm identity) only gates CID registration (creating a new vsock device for a VM); it does not gate connecting to an existing CID. So chmod/udev on the device nodes cannot close this path. Only an in-kernel hook on connect() itself can deny it.
2. Solution
A BPF LSM program attached to the socket_connect LSM hook denies connect() to AF_VSOCK port 1024 for every UID except one stored in a map (the sandlot service UID, written by the loader at attach time). All other address families and ports are passed through untouched.
Hook coverage (defense-in-depth scope). Only socket_connect is hooked: sendto()/sendmsg() on an UNCONNECTED datagram socket do not pass through that hook, so a DGRAM sender is not covered. The warden transport is a connected SOCK_STREAM (whose connect() is covered), so this is a coverage gap for the defensive layer, not a live bypass of it — a local user still cannot drive the warden without the session key.
connect(fd, {AF_VSOCK, cid, 1024})
│
security_socket_connect() ← LSM hook
│
bpf_lsm_socket_connect() ← bpf LSM (must be active!)
│
┌──────────▼──────────┐
│ vsockguard BPF prog │
│ uid == allowed_uid? │
└──────────┬──────────┘
no ↙ ↘ yes
return -EPERM return 0 (allow)
3. Files
| Path | Role |
|---|---|
network/vsockguard/vsockguard.bpf.c | BPF LSM program (C, compiled by clang) |
network/vsockguard/vsockguardctl.c | libbpf control tool: attach/detach (issues BPF_LINK_CREATE; bpftool v7.x has no LSM-link subcommand) |
network/vsockguard/vmlinux.h | generated (kernel-specific, NOT committed) from /sys/kernel/btf/vmlinux |
build/build.sh vsockguard target | compiles .bpf.o + vsockguardctl → output/ |
security/vsockguard-attach.sh [uid] | load + attach + seed allowed_uid map |
security/vsockguard-detach.sh | detach + unpin |
security/setup-bpf-lsm.sh | one-time kernel boot config to activate the bpf LSM |
security/vsockguard-verify.sh | READ-ONLY enforcement check: reads the existing pin’s allowed_uid, probes as that uid and as nobody/sandvmm, and never attaches/detaches. --self-test (only when NO pin exists) attaches a temporary one and detaches only what it created. |
e2e/ KVM test scripts (via tc_attach_vsockguard/tc_cleanup_default) | attach before the test run, detach in cleanup |
| Host Prerequisites | per-node host requirements: boot config, udev, sysctls, vsockguard attach |
The hook admits the pinned service UID and host root (root is a trust root — it can detach the program or read the session key anyway); every other UID is denied the warden port. Because /sys/fs/bpf is root-only, the non-root sandlot cannot read the pin itself: the root-run attach tool writes /run/sandswap/vsockguard.allowed_uid after a successful attach, the systemd unit re-proves the live pin in a root ExecStartPre at every start, and the sandlot validates the file’s root ownership/mode and uid. detach removes it. Build output: output/vsockguard.bpf.o + output/vsockguardctl. The attachment is pinned under /sys/fs/bpf/vsockguard/ (prog, link, allowed_uid) so it survives the ctl process exiting; detach removes all three.
4. ⚠️ Critical requirement: the bpf LSM must be active
BPF LSM hooks only fire if bpf is in the kernel’s active LSM list. Many distros (e.g. Ubuntu) ship CONFIG_LSM that omits bpf, so the bpf_lsm_socket_connect callback is never registered. In that state:
- the BPF program loads and “attaches” successfully (the link pins fine), BUT
security_socket_connect()never callsbpf_lsm_socket_connect, so- the program is a silent no-op —
connect()is never denied.
This is the dangerous failure mode: the tooling reports a successful attach while enforcement is absent.
Mitigation: vsockguardctl attach refuses to attach unless bpf is present in /sys/kernel/security/lsm, so the failure becomes an explicit error instead of silent non-enforcement. To activate bpf (one-time, same requirement as Tetragon/Falco) — full step-by-step in Host Prerequisites:
sudo bash security/setup-bpf-lsm.sh # adds bpf to the lsm= boot parameter
sudo reboot
cat /sys/kernel/security/lsm # must list 'bpf'
Then attach:
sudo bash security/vsockguard-attach.sh # pins the sandlot SERVICE account
sudo bash security/vsockguard-attach.sh --uid 985 # explicit override
With no argument the tool resolves the sandlot service account (id -u sandlot; SANDSWAP_VSOCKGUARD_UID overrides). It never falls back to SUDO_UID: the invoking operator’s uid is not the warden client, and attaching for the wrong uid silently fences the real sandlot out.
5. Verify enforcement
The sanctioned end-to-end check is security/vsockguard-verify.sh, which builds a tiny socket(AF_VSOCK)+connect() probe (vsgtest) and runs it against the EXISTING pinned program: it reads the pin’s allowed_uid and probes as that UID (allowed) and as nobody/sandvmm (denied). It never attaches or detaches a production pin. On a host with no pin, the explicit --self-test mode attaches a temporary program, probes it, and detaches only the pin it created; it refuses to run when a production pin is present. To check by hand after an attach (with bpf LSM active):
# allowed uid (the one passed to attach) → not denied (proceeds to ECONNREFUSED,
# since nothing listens at host CID 2 — the point is it was NOT EPERM)
sudo -u '#<allowed-uid>' /tmp/vsgtest 2 1024 # → ENODEV or ECONNREFUSED
# any other uid → DENIED by vsockguard
sudo -u nobody /tmp/vsgtest 2 1024 # → EPERM ("✅ DENIED by vsockguard")
# the VMM identity (sandvmm, 987) → DENIED — it must never reach the warden port
sudo -u sandvmm /tmp/vsgtest 2 1024 # → EPERM
EPERM/EACCES = hook fired and denied; any other errno on the non-allowed UID = the hook is NOT firing (re-check the bpf LSM list). The sanctioned vsockguard-verify.sh runs exactly these three probes — root allowed, nobody denied, sandvmm denied — as its pass/fail gate.
6. Operational notes
- Root can always bypass. Root can
bpftool link detachthe program orcat /proc/<vmm_pid>/mem. This is consistent with the KVM trust model: a user who controls the hypervisor host owns all VMs. vsockguard closes the non-privileged local user path only. - Guest-to-guest is neither blocked by this filter nor routed by the host — measured, not assumed. The hook keys on the host UID, and a guest’s
connect()carries no host UID, so this filter is not what confines a guest. Measured on the shipped stack: a guest’sconnect()to another VM’s CID times out (with or without the host’svsock_loopbackloaded), and to its own CID — the warden’s real port — it failsENODEV, because the shipped guest kernel has no vsock loopback transport. Both are kernel/config properties, not designed controls — which is why the raw protocol now authenticates every frame independently of the transport: each VM has a 32-byte session key (minted by the sandlot at creation and delivered over the first host connection viaOP_KEY_DELIVERY; never on the kernel command line), every request carries a nonce + HMAC-SHA256 tag verified before the body is read, and every response carries a tag bound to that nonce. vsockguard is therefore defense-in-depth, not the boundary: a kernel/transport change that made the port reachable in-guest would still meet the per-frame authentication (Isolation and Swap, “The warden’s vsock port”). - No fail-open window in the attach flow. The ctl seeds
allowed_uidBEFORE the LSM link attaches, and the single-entry map exists from load time — so the program’s!allowedbranch is defensive and unreachable: the configured UID governs every decision from the first one. (The old “briefly fail-open before the loader seeds” wording described an earlier design, not this one.) - Pass the sandlot UID explicitly. With no argument,
vsockguard-attach.shallows the invoking user’s UID (SUDO_UID) — convenient on an interactive development machine, wrong on a node: the sandlot service account would be denied every warden connect (a visible failure, not a security hole). Production deploys and the e2e harness always pass the explicit UID (985). - Idempotent.
vsockguard-attach.shdetaches any prior attachment first;vsockguard-detach.shis a no-op if not attached.