[Buildroot] [PATCH v2 01/24] package/python3: use upstream build system to disable berkeleydb module
Arnout Vandecappelle
arnout at mind.be
Tue Nov 7 08:50:41 UTC 2023
On 06/11/2023 21:20, Adam Duskett wrote:
> Ah, I missed that comment.
>
> This would be up to you and Yann to deliberate on. I was told by Yann to run a
> `git am` on Bernds patch
> series and apply it to the beginning of the series. It's a good idea as it makes
> migrating to Python 3.12.0
> easier in the case we need to revert, as the series has been thoroughly tested
> on 3.11 as well.
In that case, I disagree with Yann on this point :-)
The "use upstream build system" patches have little value by themselves;
they're only useful to make updating the package easier. So it's totally OK to
also revert all of those if we decide to revert the version bump.
So the overriding concern in that case should be ease of review, I think. And
I expect that it will be a _lot_ easier to review if it's part of the version
bump, because then it's just "use the upstream way of specifying features to
disable, and drop the now-redundant patches". With the individual "use upstream
build system" patches we have to check for every patch if it really does what it
is supposed to do.
Now, this isn't sufficient reason to ask you to spin a v3, it should be easy
enough to do the squashing while applying. That said, it's still a bit of work,
so we definitely won't complain if you spin a v3 :-)
But let's first wait if Yann has a counter-argument to me.
Regards,
Arnout
>
> Adam Duskett
>
> Senior Embedded Systems Developer
>
> M. +1208-515-8102
>
> adam.duskett at amarulasolutions.com <mailto:adam.duskett at amarulasolutions.com>
>
> __________________________________
>
>
> Amarula Solutions BV
>
> Joop Geesinkweg 125, 1114 AB, Amsterdam, NL
>
> T. +31 (0)85 111 9170
> info at amarulasolutions.com <mailto:info at amarulasolutions.com>
>
> www.amarulasolutions.com <http://www.amarulasolutions.com/>
>
>
>
>
> On Sun, Nov 5, 2023 at 2:15 PM Arnout Vandecappelle <arnout at mind.be
> <mailto:arnout at mind.be>> wrote:
>
> Hi Adam,
>
> On 05/11/2023 21:25, Adam Duskett wrote:
> > From: Bernd Kuhls <bernd at kuhls.net <mailto:bernd at kuhls.net>>
> >
> > Backport patch 0018 from python 3.12 to enhance build system.
> >
> > Signed-off-by: Bernd Kuhls <bernd at kuhls.net <mailto:bernd at kuhls.net>>
> > Reviewed-by: Adam Duskett <aduskett at gmail.com <mailto:aduskett at gmail.com>>
> > Tested-by: Adam Duskett <aduskett at gmail.com <mailto:aduskett at gmail.com>>
> > Signed-off-by: Adam Duskett <adam.duskett at amarulasolutions.com
> <mailto:adam.duskett at amarulasolutions.com>>
>
> I made the comment in v1 that this and all other "use upstream build system"
> patches should be squashed with the version bump. Did you miss that comment, or
> do you disagree?
>
> Regards,
> Arnout
>
> > ---
> > .checkpackageignore | 1 -
> > ...18-Port-_dbm-module-to-PY_STDLIB_MOD.patch | 293 ++++++++++++++++++
> > ...ion-to-disable-the-berkeleydb-module.patch | 30 --
> > package/python3/python3.mk <http://python3.mk> | 2 +-
> > 4 files changed, 294 insertions(+), 32 deletions(-)
> > create mode 100644
> package/python3/0018-Port-_dbm-module-to-PY_STDLIB_MOD.patch
> > delete mode 100644
> package/python3/0030-Add-an-option-to-disable-the-berkeleydb-module.patch
> >
> > diff --git a/.checkpackageignore b/.checkpackageignore
> > index a126d93901..be46e46c04 100644
> > --- a/.checkpackageignore
> > +++ b/.checkpackageignore
> > @@ -1166,7 +1166,6 @@
> package/python3/0026-python-config.sh-don-t-reassign-prefix.patch Upstream
> > package/python3/0027-Add-an-option-to-disable-uuid-module.patch Upstream
> > package/python3/0028-fix-building-on-older-distributions.patch Upstream
> >
> package/python3/0029-configure.ac-fixup-CC-print-multiarch-output-for-mus.patch Upstream
> >
> -package/python3/0030-Add-an-option-to-disable-the-berkeleydb-module.patch
> Upstream
> >
> package/python3/0031-lib-crypt-uClibc-ng-doesn-t-set-errno-when-encryptio.patch Upstream
> > package/qemu/0001-tests-fp-disable-fp-bench-build-by-default.patch Upstream
> >
> package/qemu/0002-softmmu-qemu-seccomp.c-add-missing-header-for-CLONE_.patch Upstream
> > diff --git a/package/python3/0018-Port-_dbm-module-to-PY_STDLIB_MOD.patch
> b/package/python3/0018-Port-_dbm-module-to-PY_STDLIB_MOD.patch
> > new file mode 100644
> > index 0000000000..92d2594eef
> > --- /dev/null
> > +++ b/package/python3/0018-Port-_dbm-module-to-PY_STDLIB_MOD.patch
> > @@ -0,0 +1,293 @@
> > +From ec5e253556875640b1ac514e85c545346ac3f1e0 Mon Sep 17 00:00:00 2001
> > +From: Christian Heimes <christian at python.org <mailto:christian at python.org>>
> > +Date: Fri, 1 Jul 2022 21:48:38 +0200
> > +Subject: [PATCH] gh-90005: Port _dbm module to PY_STDLIB_MOD (GH-94433)
> > +
> > +Upstream:
> https://github.com/python/cpython/commit/ec5e253556875640b1ac514e85c545346ac3f1e0 <https://github.com/python/cpython/commit/ec5e253556875640b1ac514e85c545346ac3f1e0>
> > +
> > +[Bernd: backported to 3.11.4]
> > +Signed-off-by: Bernd Kuhls <bernd at kuhls.net <mailto:bernd at kuhls.net>>
> > +---
> > + ...2-06-30-09-57-39.gh-issue-90005.9-pQyR.rst | 1 +
> > + Modules/Setup.stdlib.in <http://Setup.stdlib.in>
> | 2 +-
> > + configure | 295 ++++++++++++------
> > + configure.ac <http://configure.ac> |
> 100 ++++--
> > + pyconfig.h.in <http://pyconfig.h.in> |
> 6 -
> > + setup.py | 72 +----
> > + 6 files changed, 278 insertions(+), 198 deletions(-)
> > + create mode 100644
> Misc/NEWS.d/next/Build/2022-06-30-09-57-39.gh-issue-90005.9-pQyR.rst
> > +
> > +diff --git
> a/Misc/NEWS.d/next/Build/2022-06-30-09-57-39.gh-issue-90005.9-pQyR.rst
> b/Misc/NEWS.d/next/Build/2022-06-30-09-57-39.gh-issue-90005.9-pQyR.rst
> > +new file mode 100644
> > +index 0000000000000..9b14f767847da
> > +--- /dev/null
> > ++++ b/Misc/NEWS.d/next/Build/2022-06-30-09-57-39.gh-issue-90005.9-pQyR.rst
> > +@@ -0,0 +1 @@
> > ++``_dbm`` module dependencies are now detected by configure.
> > +diff --git a/Modules/Setup.stdlib.in <http://Setup.stdlib.in>
> b/Modules/Setup.stdlib.in <http://Setup.stdlib.in>
> > +index a199aefc51011..ad34f85e25451 100644
> > +--- a/Modules/Setup.stdlib.in <http://Setup.stdlib.in>
> > ++++ b/Modules/Setup.stdlib.in <http://Setup.stdlib.in>
> > +@@ -68,7 +68,7 @@
> > +
> > + # dbm/gdbm
> > + # dbm needs either libndbm, libgdbm_compat, or libdb 5.x
> > +-#@MODULE__DBM_TRUE at _dbm _dbmmodule.c
> > ++ at MODULE__DBM_TRUE@_dbm _dbmmodule.c
> > + # gdbm module needs -lgdbm
> > + @MODULE__GDBM_TRUE at _gdbm _gdbmmodule.c
> > +
> > +diff --git a/configure.ac <http://configure.ac> b/configure.ac
> <http://configure.ac>
> > +index 12ae2ae8d87eb..b03ead3bdefa0 100644
> > +--- a/configure.ac <http://configure.ac>
> > ++++ b/configure.ac <http://configure.ac>
> > +@@ -3956,17 +3956,30 @@ WITH_SAVE_ENV([
> > + ], [have_gdbm=no])
> > + ])
> > +
> > +-# check for _dbmmodule.c dependencies
> > ++dnl check for _dbmmodule.c dependencies
> > ++dnl ndbm, gdbm_compat, libdb
> > + AC_CHECK_HEADERS([ndbm.h], [
> > +- LIBS_SAVE="$LIBS"
> > +- AC_CHECK_LIB([ndbm], [dbm_open])
> > +- LIBS="$LIBS_SAVE"
> > +- AC_CHECK_LIB([gdbm_compat], [dbm_open])
> > +- LIBS="$LIBS_SAVE"
> > ++ WITH_SAVE_ENV([
> > ++ AC_SEARCH_LIBS([dbm_open], [ndbm gdbm_compat])
> > ++ ])
> > + ])
> > +
> > +-# "gdbm-ndbm.h" and "gdbm/ndbm.h" are both normalized to "gdbm_ndbm_h"
> > +-# unset ac_cv_header_gdbm_ndbm_h to prevent false positive cache hits.
> > ++AC_MSG_CHECKING([for ndbm presence and linker args])
> > ++AS_CASE([$ac_cv_search_dbm_open],
> > ++ [*ndbm*|*gdbm_compat*], [
> > ++ dbm_ndbm="$ac_cv_search_dbm_open"
> > ++ have_ndbm=yes
> > ++ ],
> > ++ [none*], [
> > ++ dbm_ndbm=""
> > ++ have_ndbm=yes
> > ++ ],
> > ++ [no], [have_ndbm=no]
> > ++)
> > ++AC_MSG_RESULT([$have_ndbm ($dbm_ndbm)])
> > ++
> > ++dnl "gdbm-ndbm.h" and "gdbm/ndbm.h" are both normalized to "gdbm_ndbm_h"
> > ++dnl unset ac_cv_header_gdbm_ndbm_h to prevent false positive cache hits.
> > + AS_UNSET([ac_cv_header_gdbm_ndbm_h])
> > + AC_CACHE_VAL([ac_cv_header_gdbm_slash_ndbm_h], [
> > + AC_CHECK_HEADER(
> > +@@ -3991,26 +4004,26 @@ AS_VAR_IF([ac_cv_header_gdbm_dash_ndbm_h], [yes], [
> > + AS_UNSET([ac_cv_header_gdbm_ndbm_h])
> > +
> > + if test "$ac_cv_header_gdbm_slash_ndbm_h" = yes -o
> "$ac_cv_header_gdbm_dash_ndbm_h" = yes; then
> > +- LIBS_SAVE="$LIBS"
> > +- AC_CHECK_LIB([gdbm_compat], [dbm_open])
> > +- LIBS="$LIBS_SAVE"
> > ++ WITH_SAVE_ENV([
> > ++ AC_SEARCH_LIBS([dbm_open], [gdbm_compat])
> > ++ ])
> > + fi
> > +
> > + # Check for libdb >= 5 with dbm_open()
> > + # db.h re-defines the name of the function
> > + AC_CHECK_HEADERS([db.h], [
> > + AC_CACHE_CHECK([for libdb], [ac_cv_have_libdb], [
> > +- LIBS_SAVE="$LIBS"
> > +- LIBS="$LIBS -ldb"
> > +- AC_LINK_IFELSE([AC_LANG_PROGRAM([
> > +- #define DB_DBM_HSEARCH 1
> > +- #include <db.h>
> > +- #if DB_VERSION_MAJOR < 5
> > +- #error "dh.h: DB_VERSION_MAJOR < 5 is not supported."
> > +- #endif
> > +- ], [DBM *dbm = dbm_open(NULL, 0, 0)])
> > +- ], [ac_cv_have_libdb=yes], [ac_cv_have_libdb=no])
> > +- LIBS="$LIBS_SAVE"
> > ++ WITH_SAVE_ENV([
> > ++ LIBS="$LIBS -ldb"
> > ++ AC_LINK_IFELSE([AC_LANG_PROGRAM([
> > ++ #define DB_DBM_HSEARCH 1
> > ++ #include <db.h>
> > ++ #if DB_VERSION_MAJOR < 5
> > ++ #error "dh.h: DB_VERSION_MAJOR < 5 is not supported."
> > ++ #endif
> > ++ ], [DBM *dbm = dbm_open(NULL, 0, 0)])
> > ++ ], [ac_cv_have_libdb=yes], [ac_cv_have_libdb=no])
> > ++ ])
> > + ])
> > + AS_VAR_IF([ac_cv_have_libdb], [yes], [
> > + AC_DEFINE([HAVE_LIBDB], [1], [Define to 1 if you have the `db'
> library (-ldb).])
> > +@@ -4018,7 +4031,7 @@ AC_CHECK_HEADERS([db.h], [
> > + ])
> > +
> > + # Check for --with-dbmliborder
> > +-AC_MSG_CHECKING(for --with-dbmliborder)
> > ++AC_MSG_CHECKING([for --with-dbmliborder])
> > + AC_ARG_WITH(dbmliborder,
> > + AS_HELP_STRING([--with-dbmliborder=db1:db2:...], [override
> order to check db backends for dbm; a valid value is a colon separated
> string with the backend names `ndbm', `gdbm' and `bdb'.]),
> > + [], [with_dbmliborder=gdbm:ndbm:bdb])
> > +@@ -4038,7 +4051,42 @@ IFS=$as_save_IFS
> > + AS_VAR_IF([with_dbmliborder], [error], [
> > + AC_MSG_ERROR([proper usage is --with-dbmliborder=db1:db2:...
> (gdbm:ndbm:bdb)])
> > + ])
> > +-AC_MSG_RESULT($with_dbmliborder)
> > ++AC_MSG_RESULT([$with_dbmliborder])
> > ++
> > ++AC_MSG_CHECKING([for _dbm module CFLAGS and LIBS])
> > ++have_dbm=no
> > ++as_save_IFS=$IFS
> > ++IFS=:
> > ++for db in $with_dbmliborder; do
> > ++ case "$db" in
> > ++ ndbm)
> > ++ if test "$have_ndbm" = yes; then
> > ++ DBM_CFLAGS="-DUSE_NDBM"
> > ++ DBM_LIBS="$dbm_ndbm"
> > ++ have_dbm=yes
> > ++ break
> > ++ fi
> > ++ ;;
> > ++ gdbm)
> > ++ if test "$have_gdbm_compat" = yes; then
> > ++ DBM_CFLAGS="-DUSE_GDBM_COMPAT"
> > ++ DBM_LIBS="-lgdbm_compat"
> > ++ have_dbm=yes
> > ++ break
> > ++ fi
> > ++ ;;
> > ++ bdb)
> > ++ if test "$ac_cv_have_libdb" = yes; then
> > ++ DBM_CFLAGS="-DUSE_BERKDB"
> > ++ DBM_LIBS="-ldb"
> > ++ have_dbm=yes
> > ++ break
> > ++ fi
> > ++ ;;
> > ++ esac
> > ++done
> > ++IFS=$as_save_IFS
> > ++AC_MSG_RESULT([$DBM_CFLAGS $DBM_LIBS])
> > +
> > + # Templates for things AC_DEFINEd more than once.
> > + # For a single AC_DEFINE, no template is needed.
> > +@@ -6940,7 +6988,9 @@ PY_STDLIB_MOD([_ctypes],
> > + dnl PY_STDLIB_MOD([_curses], [], [], [], [])
> > + dnl PY_STDLIB_MOD([_curses_panel], [], [], [], [])
> > + PY_STDLIB_MOD([_decimal], [], [], [$LIBMPDEC_CFLAGS], [$LIBMPDEC_LDFLAGS])
> > +-dnl PY_STDLIB_MOD([_dbm], [], [], [], [])
> > ++PY_STDLIB_MOD([_dbm],
> > ++ [test -n "$with_dbmliborder"], [test "$have_dbm" != "no"],
> > ++ [$DBM_CFLAGS], [$DBM_LIBS])
> > + PY_STDLIB_MOD([_gdbm],
> > + [test "$have_gdbm_dbmliborder" = yes], [test "$have_gdbm" = yes],
> > + [$GDBM_CFLAGS], [$GDBM_LIBS])
> > +diff --git a/pyconfig.h.in <http://pyconfig.h.in> b/pyconfig.h.in
> <http://pyconfig.h.in>
> > +index 15933e75b1b07..b05ddd41c2bba 100644
> > +--- a/pyconfig.h.in <http://pyconfig.h.in>
> > ++++ b/pyconfig.h.in <http://pyconfig.h.in>
> > +@@ -640,18 +640,12 @@
> > + /* Define to 1 if you have the `dld' library (-ldld). */
> > + #undef HAVE_LIBDLD
> > +
> > +-/* Define to 1 if you have the `gdbm_compat' library (-lgdbm_compat). */
> > +-#undef HAVE_LIBGDBM_COMPAT
> > +-
> > + /* Define to 1 if you have the `ieee' library (-lieee). */
> > + #undef HAVE_LIBIEEE
> > +
> > + /* Define to 1 if you have the <libintl.h> header file. */
> > + #undef HAVE_LIBINTL_H
> > +
> > +-/* Define to 1 if you have the `ndbm' library (-lndbm). */
> > +-#undef HAVE_LIBNDBM
> > +-
> > + /* Define to build the readline module. */
> > + #undef HAVE_LIBREADLINE
> > +
> > +diff --git a/setup.py b/setup.py
> > +index 843ec35effe10..cc11dedee1b2e 100644
> > +--- a/setup.py
> > ++++ b/setup.py
> > +@@ -1163,77 +1163,7 @@ def detect_crypt(self):
> > + self.addext(Extension('_crypt', ['_cryptmodule.c']))
> > +
> > + def detect_dbm_gdbm(self):
> > +- # Modules that provide persistent dictionary-like semantics.
> You will
> > +- # probably want to arrange for at least one of them to be
> available on
> > +- # your machine, though none are defined by default because of
> library
> > +- # dependencies. The Python module dbm/__init__.py provides an
> > +- # implementation independent wrapper for these; dbm/dumb.py
> provides
> > +- # similar functionality (but slower of course) implemented in
> Python.
> > +-
> > +- dbm_setup_debug = False # verbose debug prints from this script?
> > +- dbm_order = ['gdbm']
> > +-
> > +- # libdb, gdbm and ndbm headers and libraries
> > +- have_ndbm_h = sysconfig.get_config_var("HAVE_NDBM_H")
> > +- have_gdbm_ndbm_h = sysconfig.get_config_var("HAVE_GDBM_NDBM_H")
> > +- have_gdbm_dash_ndbm_h =
> sysconfig.get_config_var("HAVE_GDBM_DASH_NDBM_H")
> > +- have_libndbm = sysconfig.get_config_var("HAVE_LIBNDBM")
> > +- have_libgdbm_compat =
> sysconfig.get_config_var("HAVE_LIBGDBM_COMPAT")
> > +- have_libdb = sysconfig.get_config_var("HAVE_LIBDB")
> > +-
> > +- # The standard Unix dbm module:
> > +- if not CYGWIN:
> > +- config_args = [arg.strip("'")
> > +- for arg in
> sysconfig.get_config_var("CONFIG_ARGS").split()]
> > +- dbm_args = [arg for arg in config_args
> > +- if arg.startswith('--with-dbmliborder=')]
> > +- if dbm_args:
> > +- dbm_order = [arg.split('=')[-1] for arg in
> dbm_args][-1].split(":")
> > +- else:
> > +- dbm_order = "gdbm:ndbm:bdb".split(":")
> > +- dbmext = None
> > +- for cand in dbm_order:
> > +- if cand == "ndbm":
> > +- if have_ndbm_h:
> > +- # Some systems have -lndbm, others have
> -lgdbm_compat,
> > +- # others don't have either
> > +- if have_libndbm:
> > +- ndbm_libs = ['ndbm']
> > +- elif have_libgdbm_compat:
> > +- ndbm_libs = ['gdbm_compat']
> > +- else:
> > +- ndbm_libs = []
> > +- if dbm_setup_debug: print("building dbm using
> ndbm")
> > +- dbmext = Extension(
> > +- '_dbm', ['_dbmmodule.c'],
> > +- define_macros=[('USE_NDBM', None)],
> > +- libraries=ndbm_libs
> > +- )
> > +- break
> > +- elif cand == "gdbm":
> > +- # dbm_open() is provided by libgdbm_compat, which
> wraps libgdbm
> > +- if have_libgdbm_compat and (have_gdbm_ndbm_h or
> have_gdbm_dash_ndbm_h):
> > +- if dbm_setup_debug: print("building dbm using
> gdbm")
> > +- dbmext = Extension(
> > +- '_dbm', ['_dbmmodule.c'],
> > +- define_macros=[('USE_GDBM_COMPAT', None)],
> > +- libraries=['gdbm_compat']
> > +- )
> > +- break
> > +- elif cand == "bdb":
> > +- if have_libdb:
> > +- if dbm_setup_debug: print("building dbm using bdb")
> > +- dbmext = Extension(
> > +- '_dbm', ['_dbmmodule.c'],
> > +- define_macros=[('USE_BERKDB', None)],
> > +- libraries=['db']
> > +- )
> > +- break
> > +- if dbmext is not None:
> > +- self.add(dbmext)
> > +- else:
> > +- self.missing.append('_dbm')
> > +-
> > ++ self.addext(Extension('_dbm', ['_dbmmodule.c']))
> > + # Anthony Baxter's gdbm module. GNU dbm(3) will require -lgdbm:
> > + self.addext(Extension('_gdbm', ['_gdbmmodule.c']))
> > +
> > diff --git
> a/package/python3/0030-Add-an-option-to-disable-the-berkeleydb-module.patch
> b/package/python3/0030-Add-an-option-to-disable-the-berkeleydb-module.patch
> > deleted file mode 100644
> > index 5fb436db59..0000000000
> > ---
> a/package/python3/0030-Add-an-option-to-disable-the-berkeleydb-module.patch
> > +++ /dev/null
> > @@ -1,30 +0,0 @@
> > -From 67e9793d070ac5c8e83abbe95b9208533ffeadd0 Mon Sep 17 00:00:00 2001
> > -From: Bernd Kuhls <bernd.kuhls at t-online.de <mailto:bernd.kuhls at t-online.de>>
> > -Date: Sat, 11 Apr 2020 22:01:40 +0200
> > -Subject: [PATCH] Add an option to disable the berkeleydb module
> > -
> > -Signed-off-by: Bernd Kuhls <bernd.kuhls at t-online.de
> <mailto:bernd.kuhls at t-online.de>>
> > ----
> > - configure.ac <http://configure.ac> | 6 ++++++
> > - 1 file changed, 6 insertions(+)
> > -
> > -diff --git a/configure.ac <http://configure.ac> b/configure.ac
> <http://configure.ac>
> > -index 841fd6732c..06c9a81f95 100644
> > ---- a/configure.ac <http://configure.ac>
> > -+++ b/configure.ac <http://configure.ac>
> > -@@ -4280,6 +4280,12 @@ if test "$UUID" = "no"; then
> > - DISABLED_EXTENSIONS="${DISABLED_EXTENSIONS} _uuid"
> > - fi
> > -
> > -+AC_ARG_ENABLE(berkeleydb,
> > -+ AS_HELP_STRING([--disable-berkeleydb], [disable berkeleydb]),
> > -+ [ if test "$enableval" = "no"; then
> > -+ DISABLED_EXTENSIONS="${DISABLED_EXTENSIONS} _dbm"
> > -+ fi])
> > -+
> > - AC_SUBST(PYDOC)
> > -
> > - AC_ARG_ENABLE(pydoc,
> > ---
> > -2.34.1
> > -
> > diff --git a/package/python3/python3.mk <http://python3.mk>
> b/package/python3/python3.mk <http://python3.mk>
> > index b9c5054a21..8166d306c5 100644
> > --- a/package/python3/python3.mk <http://python3.mk>
> > +++ b/package/python3/python3.mk <http://python3.mk>
> > @@ -72,7 +72,7 @@ endif
> > ifeq ($(BR2_PACKAGE_PYTHON3_BERKELEYDB),y)
> > PYTHON3_DEPENDENCIES += berkeleydb
> > else
> > -PYTHON3_CONF_OPTS += --disable-berkeleydb
> > +PYTHON3_CONF_ENV += py_cv_module__dbm=n/a
> > endif
> >
> > ifeq ($(BR2_PACKAGE_PYTHON3_READLINE),y)
>
More information about the buildroot
mailing list