[Buildroot] [PATCH v3] package/zfs: new package
Yann E. MORIN
yann.morin.1998 at free.fr
Tue Apr 13 21:16:01 UTC 2021
José Luis, All,
On 2021-03-17 10:52 +0100, José Luis Salvador Rufo spake thusly:
> OpenZFS is an advanced file system and volume manager which was originally
> developed for Solaris and is now maintained by the OpenZFS community. This
> repository contains the code for running OpenZFS on Linux and FreeBSD.
>
> http://zfsonlinux.org/
>
> Signed-off-by: José Luis Salvador Rufo <salvador.joseluis at gmail.com>
I did a few substancial changes:
- fix test case on how to use a pre-built toolchain
- reorder the test case config
- add test case with glibc
- drop superflous test timeout override
- only select libtirpc when C library lacks native RPC
- drop unused ZFS_MODULES variable
- drop ZFS_CPE_ID_PREFIX and ZFS_AUTORECONF_OPTS which are defaults
- drop NLS options, already set in a generic manner
- drop incomplete/improper sysvinit support
- some cosmetics
Applied to master with the above changes, thanks.
Regards,
Yann E. MORIN.
> ---
> Changes v2 -> v3:
> - Updated OpenZFS to 2.0.4
> Changes v1 -> v2:
> (suggested by Yann E. MORIN)
> - Config.in: check-package accomplished
> - Config.in: fixed Config.in dependences
> - zfs.hash: proper zfs.hash format
> - zfs.mk: added cpe for openzfs
> - zfs.mk: uclibc patch merged into upstream
> - zfs.mk: removed futile comments as "zfs needs"
> - zfs.mk: added comments about the kernel dependence
> - zfs.mk: removed autotools default arguments values
> - zfs.mk: added optional PAM dependence
> - zfs.mk: replaced inline conditionals by full explained blocks
> - zfs.mk: added comments about optionals systemd and sysv
> - zfs.mk: removed --disable-silent-rules
> - zfs.mk: reordered autoreconf
> - zfs.mk: removed futile shared, static, and both logic
> - zfs.mk: fixed optional python(2|3) support for pyzfs
> - zfs.mk: removed futile kernel fragment
> - test_zfs: replaced external toolchain by already defined bootlin
> - test_zfs: increased memory a bit to prevents OOM
> - test_zfs: added pyzfs test
>
> DEVELOPERS | 4 +
> package/Config.in | 1 +
> package/zfs/Config.in | 32 ++++++++
> package/zfs/zfs.hash | 6 ++
> package/zfs/zfs.mk | 90 +++++++++++++++++++++++
> support/testing/tests/package/test_zfs.py | 61 +++++++++++++++
> 6 files changed, 194 insertions(+)
> create mode 100644 package/zfs/Config.in
> create mode 100644 package/zfs/zfs.hash
> create mode 100644 package/zfs/zfs.mk
> create mode 100644 support/testing/tests/package/test_zfs.py
>
> diff --git a/DEVELOPERS b/DEVELOPERS
> index 6a7e88039e..71e95b5bcf 100644
> --- a/DEVELOPERS
> +++ b/DEVELOPERS
> @@ -1383,6 +1383,10 @@ F: package/libuio/
> F: package/netsniff-ng/
> F: package/rabbitmq-c/
>
> +N: José Luis Salvador Rufo <salvador.joseluis at gmail.com>
> +F: package/zfs/
> +F: support/testing/tests/package/test_zfs.py
> +
> N: Joseph Kogut <joseph.kogut at gmail.com>
> F: package/at-spi2-atk/
> F: package/at-spi2-core/
> diff --git a/package/Config.in b/package/Config.in
> index d278e78b68..60afa84c2f 100644
> --- a/package/Config.in
> +++ b/package/Config.in
> @@ -229,6 +229,7 @@ menu "Filesystem and flash utilities"
> source "package/udftools/Config.in"
> source "package/unionfs/Config.in"
> source "package/xfsprogs/Config.in"
> + source "package/zfs/Config.in"
> endmenu
>
> menu "Fonts, cursors, icons, sounds and themes"
> diff --git a/package/zfs/Config.in b/package/zfs/Config.in
> new file mode 100644
> index 0000000000..96711d681e
> --- /dev/null
> +++ b/package/zfs/Config.in
> @@ -0,0 +1,32 @@
> +comment "zfs needs a Linux kernel to be built"
> + depends on !BR2_LINUX_KERNEL
> +
> +comment "zfs needs udev /dev management"
> + depends on BR2_LINUX_KERNEL
> + depends on !BR2_PACKAGE_HAS_UDEV
> +
> +config BR2_PACKAGE_ZFS
> + bool "zfs"
> + depends on BR2_LINUX_KERNEL
> + depends on BR2_PACKAGE_HAS_UDEV
> + depends on BR2_USE_MMU # util-linux (libblkid)
> + depends on BR2_TOOLCHAIN_HAS_THREADS # libtirpc
> + select BR2_PACKAGE_UTIL_LINUX
> + select BR2_PACKAGE_UTIL_LINUX_LIBBLKID
> + select BR2_PACKAGE_UTIL_LINUX_LIBUUID
> + select BR2_PACKAGE_LIBTIRPC
> + select BR2_PACKAGE_LIBAIO
> + select BR2_PACKAGE_OPENSSL
> + select BR2_PACKAGE_ZLIB
> + help
> + OpenZFS on Linux and FreeBSD
> +
> + OpenZFS is an advanced file system and volume manager which
> + was originally developed for Solaris and is now maintained by
> + the OpenZFS community. This repository contains the code for
> + running OpenZFS on Linux and FreeBSD.
> +
> + http://zfsonlinux.org/
> +
> + This package builds the kernel modules and userland utils
> + required to operates with ZFS volumes.
> diff --git a/package/zfs/zfs.hash b/package/zfs/zfs.hash
> new file mode 100644
> index 0000000000..5a0c43d366
> --- /dev/null
> +++ b/package/zfs/zfs.hash
> @@ -0,0 +1,6 @@
> +# Locally calculated
> +sha256 7d1344c5433b91823f02c2e40b33d181fa6faf286bea5591f4b1965f23d45f6c zfs-2.0.4.tar.gz
> +
> +# Hash for license files:
> +sha256 1ffb70c33c4f79f04e947facc5c7851f289609256aacb47fc115f700427d9520 LICENSE
> +sha256 f125a37c829705eff3e6f77a9f6c58fd71060f84ab54180b98fafdf94a2021eb COPYRIGHT
> diff --git a/package/zfs/zfs.mk b/package/zfs/zfs.mk
> new file mode 100644
> index 0000000000..7e7e6081a5
> --- /dev/null
> +++ b/package/zfs/zfs.mk
> @@ -0,0 +1,90 @@
> +################################################################################
> +#
> +# zfs
> +#
> +################################################################################
> +
> +ZFS_VERSION = 2.0.4
> +ZFS_SITE = https://github.com/openzfs/zfs/releases/download/zfs-$(ZFS_VERSION)
> +ZFS_LICENSE = CDDL
> +ZFS_LICENSE_FILES = LICENSE COPYRIGHT
> +ZFS_CPE_ID_VENDOR = openzfs
> +ZFS_CPE_ID_PRODUCT = openzfs
> +ZFS_CPE_ID_PREFIX = cpe:2.3:a
> +
> +# ZFS userland tools are unfunctional without its Linux kernel modules.
> +ZFS_MODULES = zavl icp zlua znvpair spl zunicode zcommon zzstd zfs
> +ZFS_MODULE_SUBDIRS = module/avl module/icp module/lua module/nvpair module/spl module/unicode module/zcommon module/zstd module/zfs
> +
> +# These requirements will be validated by zfs/config/kernel-config-defined.m4
> +define ZFS_LINUX_CONFIG_FIXUPS
> + $(call KCONFIG_DISABLE_OPT,CONFIG_DEBUG_LOCK_ALLOC)
> + $(call KCONFIG_DISABLE_OPT,CONFIG_TRIM_UNUSED_KSYMS)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_CRYPTO_DEFLATE)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_ZLIB_DEFLATE)
> + $(call KCONFIG_ENABLE_OPT,CONFIG_ZLIB_INFLATE)
> +endef
> +
> +ZFS_AUTORECONF = YES
> +ZFS_AUTORECONF_OPTS = -fiv
> +
> +ZFS_DEPENDENCIES += zlib util-linux libtirpc libaio openssl udev
> +ZFS_CONF_OPTS = \
> + --with-linux=$(LINUX_DIR) \
> + --with-linux-obj=$(LINUX_DIR) \
> + --with-tirpc \
> + --disable-rpath
> +
> +ifeq ($(BR2_ENABLE_LOCALE),)
> +ZFS_CONF_OPTS += --enable-nls
> +else
> +ZFS_CONF_OPTS += --disable-nls
> +endif
> +
> +ifeq ($(BR2_INIT_SYSTEMD),y)
> +# Installs the optional systemd generators, units, and presets files.
> +ZFS_CONF_OPTS += --enable-systemd
> +else
> +ZFS_CONF_OPTS += --disable-systemd
> +endif
> +
> +ifeq ($(BR2_INIT_SYSV),y)
> +# Installs the optional sysvinit modules-load.d conf. file.
> +ZFS_CONF_OPTS += --enable-sysvinit
> +else
> +ZFS_CONF_OPTS += --disable-sysvinit
> +endif
> +
> +# The optional PyZFS uses different scripts depending on the python
> +# version (ex: arc_summary2 or arc_summary3).
> +ifeq ($(BR2_PACKAGE_PYTHON),y)
> +ZFS_DEPENDENCIES += python python-setuptools host-python-cffi
> +ZFS_CONF_ENV += \
> + PYTHON=$(HOST_DIR)/usr/bin/python2 \
> + PYTHON_CPPFLAGS="`$(STAGING_DIR)/usr/bin/python2-config --includes`" \
> + PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python2-config --ldflags`" \
> + PYTHON_SITE_PKG="/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages"
> +ZFS_CONF_OPTS += --enable-pyzfs
> +else ifeq ($(BR2_PACKAGE_PYTHON3),y)
> +ZFS_DEPENDENCIES += python3 python-setuptools host-python-cffi
> +ZFS_CONF_ENV += \
> + PYTHON=$(HOST_DIR)/usr/bin/python3 \
> + PYTHON_CPPFLAGS="`$(STAGING_DIR)/usr/bin/python3-config --includes`" \
> + PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --ldflags`" \
> + PYTHON_EXTRA_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --libs --embed`" \
> + PYTHON_SITE_PKG="/usr/lib/python$(PYTHON_VERSION_MAJOR)/site-packages"
> +ZFS_CONF_OPTS += --enable-pyzfs
> +else
> +ZFS_CONF_OPTS += --disable-pyzfs --without-python
> +endif
> +
> +ifeq ($(BR2_PACKAGE_LINUX_PAM),y)
> +# ZFS will check for pam_modules.h
> +ZFS_DEPENDENCIES += linux-pam
> +ZFS_CONF_ENV += --enable-pam=yes
> +else
> +ZFS_CONF_OPTS += --disable-pam
> +endif
> +
> +$(eval $(kernel-module))
> +$(eval $(autotools-package))
> diff --git a/support/testing/tests/package/test_zfs.py b/support/testing/tests/package/test_zfs.py
> new file mode 100644
> index 0000000000..6a6d2da120
> --- /dev/null
> +++ b/support/testing/tests/package/test_zfs.py
> @@ -0,0 +1,61 @@
> +import os
> +
> +from ..toolchain.test_external_bootlin import (
> + TestExternalToolchainBootlinX8664corei7UclibcStable,
> +)
> +
> +
> +class TestZfs(TestExternalToolchainBootlinX8664corei7UclibcStable):
> + config = (
> + TestExternalToolchainBootlinX8664corei7UclibcStable.config
> + + """
> + BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/x86_64/linux.config"
> + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="5.10.23"
> + BR2_LINUX_KERNEL_CUSTOM_VERSION=y
> + BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
> + BR2_LINUX_KERNEL=y
> + BR2_PACKAGE_BUSYBOX=y
> + BR2_PACKAGE_ZFS=y
> + BR2_PACKAGE_ZLIB_NG=y
> + BR2_PACKAGE_LIBRESSL=y
> + BR2_PACKAGE_PYTHON3=y
> + BR2_PACKAGE_PYTHON_SETUPTOOLS=y
> + BR2_PACKAGE_PYTHON_CFFI=y
> + BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_EUDEV=y
> + BR2_TARGET_ROOTFS_CPIO=y
> + """
> + )
> +
> + def test_run(self):
> + kernel = os.path.join(self.builddir, "images", "bzImage")
> + cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio")
> + self.emulator.boot(
> + arch="x86_64",
> + kernel=kernel,
> + kernel_cmdline=["console=ttyS0"],
> + options=["-cpu", "Nehalem", "-m", "320", "-initrd", cpio_file],
> + )
> + self.emulator.login()
> +
> + timeout = 35 * self.emulator.timeout_multiplier
> + cmds = [
> + # Init
> + "modprobe zfs",
> + "mount -o remount,size=132M /tmp",
> + "fallocate -l 64M /tmp/container1.raw",
> + "fallocate -l 64M /tmp/container2.raw",
> + "zpool create -m /pool pool raidz /tmp/container1.raw /tmp/container2.raw",
> + "dd if=/dev/urandom bs=1M count=8 of=/pool/urandom",
> + "sha256sum /pool/urandom > /tmp/urandom.sha256",
> + # Check ZFS
> + "zpool export pool",
> + "zpool import pool -d /tmp/container1.raw -d /tmp/container2.raw",
> + "dd conv=notrunc bs=1M count=32 seek=16 if=/dev/urandom of=/tmp/container1.raw",
> + "zpool scrub -w pool",
> + "sha256sum -c /tmp/urandom.sha256",
> + "zpool status -v",
> + # Check PyZFS
> + "arc_summary",
> + ]
> + for cmd in cmds:
> + self.assertRunOk(cmd, timeout=timeout)
> --
> 2.31.0
>
--
.-----------------.--------------------.------------------.--------------------.
| Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ |
| +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. |
'------------------------------^-------^------------------^--------------------'
More information about the buildroot
mailing list