Skip to content

Role: auto_updates

Fleet-wide unattended-upgrades. Applied to every host in inventory via playbooks/auto-updates.yml (hosts: all), so new LXCs/VMs inherit the policy on the next drift run with no per-playbook wiring.

Hosts: every host in inventory/hosts.yml — currently proxfold, arrstack, matrix, n8n, plex, pbs, beszel, edge, forgejo, vintage, control. Opt-out per host via auto_updates_enabled: false.

Docker containers are not in scope — Dockhand handles those on the arrstack/n8n stacks; spantaleev's playbook owns container lifecycle on the matrix VM.

Why a wrapper role

The engine is the community-maintained hifis.toolkit.unattended_upgrades (the actively-developed successor to the now-archived jnv.unattended-upgrades). We wrap it rather than reinvent it; the wrapper layer adds:

  • A namespaced variable surface (auto_updates_*) that's obvious in host_vars.
  • A minimal kernel postinst hook (/etc/kernel/postinst.d/zz-ansible-reboot-required) that populates /var/run/reboot-required when a kernel package version differs from the running kernel. Replaces update-notifier-common, which was dropped in Debian 13 trixie; the upstream successor reboot-notifier hard-depends on an MTA, which we don't want to inflict on every container. The hook's flag-file format is the canonical one, so any tooling that reads /var/run/reboot-required keeps working.
  • needrestart installed in non-interactive ('l') mode to catch the gap the home-rolled hook doesn't cover (libc6/openssl/systemd updates needing a service restart) and to flag a stale kernel even when the postinst hook missed it.
  • An opt-in Discord "reboot required" notifier — systemd timer that POSTs to the shared #homelab-ops webhook while the flag file exists.

Policy

Knob Default Rationale
Origins Debian-Security only Matches Debian and hifis defaults. Security patches land, point-release breakage stays out.
Automatic reboot off Fleet-wide. The notifier covers "something needs a reboot"; the operator picks when.
Mail off No MTA on these hosts. Signalling goes via Discord.
Blocklist empty Per-host override on proxfold pins kernels out (below).

Per-host overrides

proxfold (PVE host) — adds the Proxmox no-subscription repo to the origins pattern so PVE-security fixes fast-track. Kernels (proxmox-kernel-*, proxmox-kernel-signed-*, pve-kernel-*) are blocklisted because a hypervisor kernel rolling forward without operator intent + a planned reboot is the landmine the Proxmox forum keeps calling out. Discord nag enabled (auto_updates_notify_discord: true).

pbs (CT 105) — same Proxmox origin so PBS security fixes land. Kernel blocklist is moot inside an LXC (host kernel).

edge (CT 107, Phase 7D) — adds the CrowdSec packagecloud repo (origin=packagecloud.io/crowdsec/crowdsec,suite=any) so engine security updates flow with the daily timer. The exact Origin: string is empirically inconsistent across packagecloud repos; verify post-install with apt-cache policy crowdsec and grep ^Origin /var/lib/apt/lists/packagecloud*_InRelease, and correct the inventory line if it differs from what unattended-upgrades sees.

All other hosts — hifis defaults only. Debian-Security, nothing else.

Tasks

Task Tag
Drop kernel postinst hook → populates /var/run/reboot-required auto_updates, reboot_flag
include_role: hifis.toolkit.unattended_upgrades with translated vars auto_updates, unattended
Install reboot-required notify script (opt-in) auto_updates, notify
Install systemd service + timer (opt-in) auto_updates, notify
Enable timer (non-check only — unit doesn't exist in check mode on fresh host) auto_updates, notify

Key variables

Variable Default Purpose
auto_updates_enabled true Master per-host switch
auto_updates_origins_base Debian-Security × 2 patterns Fleet baseline origins
auto_updates_origins_extra [] Per-host additions (proxfold/pbs add Proxmox repo)
auto_updates_blocklist [] Packages apt never auto-upgrades
auto_updates_automatic_reboot false Flip per-host if you ever want unattended reboots
auto_updates_mail false —
auto_updates_notify_discord false Enable systemd-timer-driven Discord ping while /var/run/reboot-required exists
auto_updates_notify_oncalendar *-*-* 09:00:00 Timer OnCalendar spec
auto_updates_webhook_var vault_discord_webhook_homelab_ops Vault key that holds the webhook URL (shared with 5B)
auto_updates_needrestart_enabled true Install + configure needrestart with non-interactive mode
auto_updates_needrestart_mode 'l' 'l' (list-only) — never 'i' (would prompt and stall unattended-upgrades) or 'a' (auto-restart, risky on stateful daemons)
auto_updates_needrestart_kernelhints -1 on VMs/host, 0 on LXCs LXCs share the host kernel; suppress the stale-kernel hint there to keep apt output clean

Scaling to new hosts

  1. Add the host to inventory/hosts.yml.
  2. It picks up auto_updates on the next site.yml / drift run automatically.
  3. If the host has non-Debian repos that warrant extra origins or kernel pinning, drop a host_var.

Bootstrap note — collection install

The hifis.toolkit collection is declared in requirements.yml. Install (or refresh) on any control node before the first run:

ansible-galaxy collection install -r requirements.yml

Runs from CT104 (daily drift runner) and WSL (secondary control) both need this once.

"Needs reboot" signalling

Three layers, in order of coverage:

  1. Passive — kernel postinst hook (always-on, fleet-wide). Drops /var/run/reboot-required + .pkgs whenever a kernel package version differs from the running kernel. Debian's dynamic motd picks up the flag file and shows a banner on SSH login. The flag is on tmpfs so reboot clears it. LXCs don't install kernel packages, so this layer is inert there.
  2. Passive — needrestart (always-on, fleet-wide). Runs on every apt operation via /etc/apt/apt.conf.d/99needrestart. Catches the libc6/openssl/systemd case (services using deleted libraries) that the postinst hook misses — listed in apt output and via the needrestart command on login. Also independently writes /var/run/reboot-required when it detects a stale kernel, so the proxfold-style notifier layer below works regardless of which path tripped it.
  3. Active — opt-in Discord notifier. reboot-required-notify.timer fires daily at 09:00 and POSTs a Discord embed listing the pending packages while the flag file exists. Stops nagging once the host reboots. Reuses the existing vault_discord_webhook_homelab_ops webhook — no new secrets.

Active-notifier enablement table:

Host class auto_updates_notify_discord Rationale
Bare-metal / VMs (proxfold, arrstack, matrix, n8n) true Own a kernel; -security updates land via the timer; operator needs prompting
LXCs (pbs, beszel, edge, forgejo, plex, vintage, control) false (default) Share the host kernel; the postinst hook doesn't fire and there's nothing to nag about. Container restart for libc6/etc is visible in apt output via needrestart and via SSH-login motd

Coverage caveat: the active Discord notifier only fires on /var/run/reboot-required, which is written by kernel events. needrestart's "services X, Y, Z need a restart" output (the libc6/openssl case) is on-host only — visible via apt upgrade, needrestart on the CLI, and the SSH login banner, but not in Discord. To route that into Discord too would need a small parser around needrestart -b output; deliberately deferred today.

Check-mode note

The role is --check --diff clean after the first real run on each host. Before the initial install, the timer-enable task is gated on not ansible_check_mode because the unit file doesn't exist yet on a fresh host. Standard first-install pattern — same as pbs, beszel_agent, etc.