[Buildroot] [PATCH 2/3 v4] system: add option to use an overlayfs on /var on a r/o root w/ systemd
Norbert Lange
nolange79 at gmail.com
Sun Jan 8 23:26:59 UTC 2023
Am So., 25. Dez. 2022 um 23:08 Uhr schrieb Yann E. MORIN
<yann.morin.1998 at free.fr>:
>
> From: "Yann E. MORIN" <yann.morin at orange.com>
>
> While the /var factory seems to be working in most cases, there have
> been suggestions that it may be slightly and subtely borken in some
> (rare? edge?) cases, especially about symlinks.
>
> An other solution is to pre-populate /var at build time, by way of
> calling systemd-tmpfiles, and mounting an overlayfs on-top of it at
> runtime. The first part is already done, and this change focuses on
> mountig the overlayfs.
>
> This is slightly accrobatic, though, and requires a few hoops:
>
> - first, we create a tmpfs; this will server as the backing store for
> the writable part of the overlayfs;
>
> - there, we create three directories for the overlayfs:
> - lower/, upper/, and work/, to serve as the respective overlayfs
> directories,
> - note: the 'upper' and 'work' have to be on the same filesystem;
>
> - then we bind-mount /var onto lower/
>
> - eventually we mount the overlayfs
>
> We are doing that with three systemd units, with the latter having a
> dependency on the previous one:
>
> - the tmpfs is created as a standalone mount unit, first because that
> just makes sense to use systemd to do the mount, and second because
> that will make it easy gfor users to provide an actual backing store
> for /var, by just providing a dropin that overrides the What with
> the desitred backing store, and optionally adding dependencies on
> other services to create/initialise the backing store if needed.
>
> - creating the directories is done with their own service file, as
> there is no other easy way to reate them automatically. We also
> bind-mount /var onto the lower in that service file, instead of
> using a systemd mount unit. See [0] for the rationale.
>
> - finally, we mount an overlay on /var with a classic systemd mount
> unit.
>
> [0] if we had a systemd mount unit to do the bind mount, then it sould
> look like:
> # run-buildroot-mounts-var-lower.mount
> [Mount]
> What=/var
> Where=/run/buildroot/mounts/var/lower
> Options=bind
>
> and then the var.mount unit would need to have a dependency on that
> unit:
> # var.mount
> [Unit]
> After=run-buildroot-mounts-var-lower.mount
> [Mount]
> Where=/var
>
> However, the What=/var of the first unit automatically adds an implicit
> dependency on /var, and since there is a unit providing Where=/var, we
> would have run-buildroot-mounts-var-lower.mount depend on var.mount, but
> we need var.mount to depend on run-buildroot-mounts-var-lower.mount, so
> this is a circular dependency. There is no way to tell systemd no to add
> the implicit dependency. So we do the bind mont manually in the service
> unit that prepares the overlay structure.
>
> Norbert provided some systemd units as a starting point, and that was
> quite a huge help in understanding how to fit all those things together.
>
> Co-authored-by: Norbert Lange <nolange79 at gmail.com>
> Signed-off-by: Yann E. MORIN <yann.morin at orange.com>
> Cc: Norbert Lange <nolange79 at gmail.com>
> Cc: Romain Naour <romain.naour at smile.fr>
> Cc: Jérémy Rosen <jeremy.rosen at smile.fr>
> Signed-off-by: Yann E. MORIN <yann.morin.1998 at free.fr>
> ---
> .../{ => factory}/var.mount | 0
> .../overlayfs/prepare-var-overlay.service | 19 +++++++++++++
> .../overlayfs/run-buildroot-mounts-var.mount | 12 ++++++++
> .../skeleton-init-systemd/overlayfs/var.mount | 14 ++++++++++
> .../skeleton-init-systemd.mk | 28 +++++++++++++++++--
> system/Config.in | 26 ++++++++++++-----
> 6 files changed, 89 insertions(+), 10 deletions(-)
> rename package/skeleton-init-systemd/{ => factory}/var.mount (100%)
> create mode 100644 package/skeleton-init-systemd/overlayfs/prepare-var-overlay.service
> create mode 100644 package/skeleton-init-systemd/overlayfs/run-buildroot-mounts-var.mount
> create mode 100644 package/skeleton-init-systemd/overlayfs/var.mount
>
> diff --git a/package/skeleton-init-systemd/var.mount b/package/skeleton-init-systemd/factory/var.mount
> similarity index 100%
> rename from package/skeleton-init-systemd/var.mount
> rename to package/skeleton-init-systemd/factory/var.mount
> diff --git a/package/skeleton-init-systemd/overlayfs/prepare-var-overlay.service b/package/skeleton-init-systemd/overlayfs/prepare-var-overlay.service
> new file mode 100644
> index 0000000000..281aa0efb5
> --- /dev/null
> +++ b/package/skeleton-init-systemd/overlayfs/prepare-var-overlay.service
> @@ -0,0 +1,19 @@
> +[Unit]
> +Description=Variable storage overlay setup
> +ConditionPathIsSymbolicLink=!/var
> +DefaultDependencies=no
> +RequiresMountsFor=/run/buildroot/mounts/var
> +
> +[Service]
> +Type=oneshot
> +RemainAfterExit=yes
> +ExecStart=/usr/bin/mkdir -p /run/buildroot/mounts/var/lower /run/buildroot/mounts/var/upper /run/buildroot/mounts/var/work
> +
> +# Ideally, we would like to use a systemd mount unit to manage the bind
> +# mount. Unfortunately, that creates a circular dependency: such a unit
> +# would have What=/var while var.mount has Where=/var so that introduces
> +# an implicit dependency from that unit to var.mount, but var.mount
> +# would have an explicit dependency to be ordered after that unit.
> +# So we handle the bind mount manually.
> +ExecStart=/usr/bin/mount -n -o bind,private /var /run/buildroot/mounts/var/lower
> +ExecStop=/usr/bin/umount -l /run/buildroot/mounts/var/lower
> diff --git a/package/skeleton-init-systemd/overlayfs/run-buildroot-mounts-var.mount b/package/skeleton-init-systemd/overlayfs/run-buildroot-mounts-var.mount
> new file mode 100644
> index 0000000000..554975a052
> --- /dev/null
> +++ b/package/skeleton-init-systemd/overlayfs/run-buildroot-mounts-var.mount
> @@ -0,0 +1,12 @@
> +[Unit]
> +Description=Variable storage overlay tmpfs
> +ConditionPathIsSymbolicLink=!/var
> +DefaultDependencies=no
> +After=local-fs-pre.target
> +
> +[Mount]
> +What=var_overlay_tmpfs
> +Where=/run/buildroot/mounts/var
> +Type=tmpfs
> +Options=private
> +LazyUnmount=true
> diff --git a/package/skeleton-init-systemd/overlayfs/var.mount b/package/skeleton-init-systemd/overlayfs/var.mount
> new file mode 100644
> index 0000000000..812e6ce7bf
> --- /dev/null
> +++ b/package/skeleton-init-systemd/overlayfs/var.mount
> @@ -0,0 +1,14 @@
> +[Unit]
> +Description=Variable storage overlay
> +Documentation=man:file-hierarchy(7)
> +ConditionPathIsSymbolicLink=!/var
> +DefaultDependencies=no
> +After=prepare-var-overlay.service
> +BindsTo=prepare-var-overlay.service
> +
> +[Mount]
> +What=overlay_var
> +Where=/var
> +Type=overlay
> +Options=lowerdir=/run/buildroot/mounts/var/lower,upperdir=/run/buildroot/mounts/var/upper,workdir=/run/buildroot/mounts/var/work,redirect_dir=on,index=on,xino=on
> +LazyUnmount=true
> diff --git a/package/skeleton-init-systemd/skeleton-init-systemd.mk b/package/skeleton-init-systemd/skeleton-init-systemd.mk
> index fb15552f99..cc32960dbf 100644
> --- a/package/skeleton-init-systemd/skeleton-init-systemd.mk
> +++ b/package/skeleton-init-systemd/skeleton-init-systemd.mk
> @@ -33,7 +33,7 @@ endef
> # a real (but empty) directory, and the "factory files" will be copied
> # back there by the tmpfiles.d mechanism.
> ifeq ($(BR2_INIT_SYSTEMD_VAR_FACTORY),y)
> -define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
> +define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR_FACTORY
> rm -rf $(TARGET_DIR)/usr/share/factory/var
> mv $(TARGET_DIR)/var $(TARGET_DIR)/usr/share/factory/var
> mkdir -p $(TARGET_DIR)/var
> @@ -52,11 +52,33 @@ define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
> || exit 1; \
> fi; \
> done >$(TARGET_DIR)/usr/lib/tmpfiles.d/00-buildroot-var.conf
> - $(INSTALL) -D -m 0644 $(SKELETON_INIT_SYSTEMD_PKGDIR)/var.mount \
> + $(INSTALL) -D -m 0644 $(SKELETON_INIT_SYSTEMD_PKGDIR)/factory/var.mount \
> $(TARGET_DIR)/usr/lib/systemd/system/var.mount
> endef
> -SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR
> +SKELETON_INIT_SYSTEMD_ROOTFS_PRE_CMD_HOOKS += SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR_FACTORY
> endif # BR2_INIT_SYSTEMD_VAR_FACTORY
> +
> +ifeq ($(BR2_INIT_SYSTEMD_VAR_OVERLAYFS),y)
> +
> +define SKELETON_INIT_SYSTEMD_LINUX_CONFIG_FIXUPS
> + $(call KCONFIG_ENABLE_OPT,CONFIG_OVERLAY_FS)
> +endef
> +
> +define SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR_OVERLAYFS
> + $(INSTALL) -D -m 0644 \
> + $(SKELETON_INIT_SYSTEMD_PKGDIR)/overlayfs/run-buildroot-mounts-var.mount \
> + $(TARGET_DIR)/usr/lib/systemd/system/run-buildroot-mounts-var.mount
> + $(INSTALL) -D -m 0644 \
> + $(SKELETON_INIT_SYSTEMD_PKGDIR)/overlayfs/prepare-var-overlay.service \
> + $(TARGET_DIR)/usr/lib/systemd/system/prepare-var-overlay.service
> + $(INSTALL) -D -m 0644 \
> + $(SKELETON_INIT_SYSTEMD_PKGDIR)/overlayfs/var.mount \
> + $(TARGET_DIR)/usr/lib/systemd/system/var.mount
> +endef
> +SKELETON_INIT_SYSTEMD_POST_INSTALL_TARGET_HOOKS += SKELETON_INIT_SYSTEMD_PRE_ROOTFS_VAR_OVERLAYFS
> +
> +endif # BR2_INIT_SYSTEMD_VAR_OVERLAYFS
> +
> endif # BR2_TARGET_GENERIC_REMOUNT_ROOTFS_RW
>
> ifeq ($(BR2_INIT_SYSTEMD_POPULATE_TMPFILES),y)
> diff --git a/system/Config.in b/system/Config.in
> index 87df031545..5c9b41a8d3 100644
> --- a/system/Config.in
> +++ b/system/Config.in
> @@ -164,6 +164,14 @@ choice
> Select how Buildroot provides a read-write /var when the
> rootfs is not remounted read-write.
>
> + Note: Buildroot uses a tmpfs, either as a mount point or as
> + the upper of an overlayfs, so as to at least make the system
> + bootable out of the box; mounting a filesystem from actual
> + storage is left to the integration, as it is too specific and
> + may need preparatory work like partitionning a device and/or
> + formatting a filesystem first, which falls out of the scope
> + of Buildroot.
> +
> config BR2_INIT_SYSTEMD_VAR_FACTORY
> bool "build a factory to populate a tmpfs"
> help
> @@ -176,17 +184,21 @@ config BR2_INIT_SYSTEMD_VAR_FACTORY
> It probably does not play very well with triggering a call
> to systemd-tmpfiles at build time (below).
>
> - Note: Buildroot mounts a tmpfs on /var to at least make the
> - system bootable out of the box; mounting a filesystem from
> - actual storage is left to the integration, as it is too
> - specific and may need preparatory work like partitionning a
> - device and/or formatting a filesystem first, so that falls
> - out of the scope of Buildroot.
> -
> To use persistent storage, provide a systemd dropin for the
> var.mount unit, that overrides the What and Type, and possibly
> the Options and After, fields.
>
> +config BR2_INIT_SYSTEMD_VAR_OVERLAYFS
> + bool "mount an overlayfs backed by a tmpfs"
> + select BR2_INIT_SYSTEMD_POPULATE_TMPFILES
> + help
> + Mount an overlayfs on /var, with the upper as a tmpfs.
> +
> + To use a persistent storage, provide a systemd dropin for the
> + run-buildroot-mounts-var.mount unit, that overrides the What
> + and Type, and possibly Options, fields, and adds necessary
> + dependencies on other services.
> +
> config BR2_INIT_SYSTEMD_VAR_NONE
> bool "do nothing"
> help
> --
> 2.25.1
>
Just a heads up that I worked on a new version with several
cleanups/improvements,
still need to rework it into a proper patch with explanations, hope I will
finish it next weekend.
Norbert
More information about the buildroot
mailing list