[Buildroot] [PATCH v2 2/2] utils/scanpypi: add flit package support

[James Hilliard]

On Wed, Aug 3, 2022 at 3:46 PM Thomas Petazzoni
<thomas.petazzoni at bootlin.com> wrote:
>
> On Sun, 13 Mar 2022 11:13:55 -0600
> James Hilliard <james.hilliard1 at gmail.com> wrote:
>
> > +def toml_load(f):
> > +    with open(f, 'rb') as fh:
> > +        ex = None
> > +
> > +        # Try regular tomli first
> > +        try:
> > +            from tomli import load
> > +            return load(fh)
> > +        except ImportError as e:
> > +            ex = e
> > +
> > +        # Try pip's vendored tomli
> > +        try:
> > +            from pip._vendor.tomli import load
> > +            try:
> > +                return load(fh)
> > +            except TypeError:
> > +                # Fallback to handle older version
> > +                try:
> > +                    fh.seek(0)
> > +                    w = io.TextIOWrapper(fh, encoding="utf8", newline="")
> > +                    return load(w)
> > +                finally:
> > +                    w.detach()
> > +        except ImportError as e:
> > +            pass
> > +
> > +        # Try regular toml last
> > +        try:
> > +            from toml import load
> > +            fh.seek(0)
> > +            w = io.TextIOWrapper(fh, encoding="utf8", newline="")
> > +            try:
> > +                return load(w)
> > +            finally:
> > +                w.detach()
> > +        except ImportError:
> > +            pass
> > +
> > +        print('This package needs tomli')
> > +        raise ex
>
> I'm confused by how "ex" gets used here. It's initially none, and it's
> set to the ImportError exception if the tomli module couldn't be used.
>
> >      def get_requirements(self, pkg_folder):
> >          """
> >          Retrieve dependencies from the metadata found in the setup.py script of
> > @@ -694,9 +767,12 @@ def main():
> >              except ImportError as err:
> >                  if 'buildutils' in str(err):
> >                      print('This package needs buildutils')
> > +                    continue
> >                  else:
> > -                    raise
> > -                continue
> > +                    try:
> > +                        package.load_pyproject()
> > +                    except Exception as e:
> > +                        raise
>
> I really don't like the construction here. We're doing this:
>
>         try:
>                 ... do an attempt with traditional setup.py ...
>         expect ImportError as err:
>                 ...
>                 try:
>                         ... do an attempt with pyproject.toml ...
>                 expect Exception as e:
>                         ...
>
> So, if get a third and then a fourth method of Python module packaging,
> we will end up with:

All new python packaging methods are supposed to contain a pyproject.toml
file per pep517 so I think we can probably defer refactoring this for now.

The only supported packaging method in the absence of a pyproject.toml
will be setuptools(distutils is effectively being moved inside of setuptools
to avoid breaking older packages that are not migrated to setuptools, so
once distutils is removed from python our current distutils packages will
need to be moved to setuptools infrastructure, but that should more or
less just work automatically).

>
>         try:
>                 ... do an attempt with traditional setup.py ...
>         expect ImportError as err:
>                 ...
>                 try:
>                         ... do an attempt with pyproject.toml ...
>                 expect Exception as e:
>                         ...
>                         try:
>                                 ... do an attempt with another method
>                         except:
>                                 try:
>                                         ... do an attempt with another method
>                                 except:
>
> Clearly ugly. Can we have a better construct? I would know how to do
> that in C, but I'm not very good at Python-ic constructs.
>
> Thanks!
>
> Thomas
> --
> Thomas Petazzoni, co-owner and CEO, Bootlin
> Embedded Linux and Kernel engineering and training
> https://bootlin.com