[Buildroot] How to install custom sdk in buildroot?

Tue Feb 14 07:27:34 UTC 2023

Hello James,

Thanks for the update.

awscrt also has precompiled wheels
<https://pypi.org/project/awscrt/#files> such
as
awscrt-0.16.10-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Can Buildroot use them instead of cross compiling? If not, is it possible
to extract it and install manually?

Thanks in advance

On Thu, 9 Feb 2023 at 21:12, James Hilliard <james.hilliard1 at gmail.com>
wrote:

> On Thu, Feb 9, 2023 at 5:10 AM Sourabh Hegde <hrsourabh011 at gmail.com>
> wrote:
> >
> > Hello Maxim,
> >
> > While doing make with new Buildroot-packages I getting:
> >
> > [ 51%] Building ASM object
> aws-lc/crypto/CMakeFiles/crypto_objects.dir/hrss/asm/poly_rq_mul.S.o
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:
> Assembler messages:
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:304:
> Error: unknown pseudo-op: `.att_syntax'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:307:
> Error: unknown mnemonic `push' -- `push %rbp'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:309:
> Error: bad register expression
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:310:
> Error: unknown mnemonic `movq' -- `movq %rsp,%rbp'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:311:
> Error: bad register expression
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:312:
> Error: unknown mnemonic `push' -- `push %r12'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:313:
> Error: bad register expression
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:321:
> Error: operand 1 must be an integer register -- `mov %rcx,%r8'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:322:
> Error: unknown mnemonic `addq' -- `addq $6144+12288+512+9408+32,%r8'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:323:
> Error: operand 1 must be an integer register -- `mov %r8,%rax'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:324:
> Error: unknown mnemonic `subq' -- `subq $6144,%r8'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:325:
> Error: operand 1 must be an integer register -- `mov %r8,%r11'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:326:
> Error: unknown mnemonic `subq' -- `subq $12288,%r8'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:327:
> Error: operand 1 must be an integer register -- `mov %r8,%r12'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:328:
> Error: unknown mnemonic `subq' -- `subq $512,%r8'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:329:
> Error: unknown mnemonic `vmovdqa' -- `vmovdqa const3(%rip),%ymm3'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:330:
> Error: unknown mnemonic `vmovdqu' -- `vmovdqu 0(%rsi),%ymm0'
> >
> /home/amazone/output/build/python-awscrt-0.16.10/crt/aws-lc/crypto/hrss/asm/poly_rq_mul.S:331:
> Error: unknown mnemonic `vmovdqu' -- `vmovdqu 88(%rsi),%ymm1'
> > .
> > .
> > .
> > Any idea what it is complaining about?
>
> The problem here is that while the python awscrt package uses setuptools
> it's not a normal python setuptools package, the setup.py installs both a
> normal
> setuptools package and a vendored cmake package(meta package really).
>
> See cmake build wrapper functionality here in the setuptools setup.py file:
> https://github.com/awslabs/aws-crt-python/blob/v0.16.10/setup.py#L156-L272
>
> This is the cmake meta project root which is vendored inside of awscrt:
> https://github.com/awslabs/aws-crt-python/tree/v0.16.10/crt
>
> See meta cmakelists build:
> https://github.com/awslabs/aws-crt-python/blob/v0.16.10/crt/CMakeLists.txt
>
> This meta project is made up of multiple subprojects:
> aws-c-auth
> aws-c-cal
> aws-c-common
> aws-c-compression
> aws-c-event-stream
> aws-checksums
> aws-c-http
> aws-c-io
> aws-c-mqtt
> aws-c-s3
> aws-c-sdkutils
> aws-lc
> s2n
>
> In buildroot we expect cmake packages to use our cmake infrastructure
> as it will set up the cross compilation environment correctly while the
> awscrt setuptools cmake wrapper does not.
>
> I confirmed that the cmake packages build correctly when using our
> cmake infrastructure:
>
> ################################################################################
> #
> # python-awscrt
> #
>
> ################################################################################
>
> PYTHON_AWSCRT_VERSION = 0.16.10
> PYTHON_AWSCRT_SOURCE = awscrt-$(PYTHON_AWSCRT_VERSION).tar.gz
> PYTHON_AWSCRT_SITE =
>
> https://files.pythonhosted.org/packages/ae/98/dda092078f2d47bad08e184ab9d364b8f89166497062bc1cf811c0bbbde2
> PYTHON_AWSCRT_SUBDIR = crt
>
> $(eval $(cmake-package))
>
> See:
>
> https://buildroot.org/downloads/manual/manual.html#_infrastructure_for_cmake_based_packages
>
> So what I think you need to do is create buildroot cmake packages for all
> these subprojects using buildroot's cmake infrastructure:
> https://github.com/awslabs/aws-c-auth
> https://github.com/awslabs/aws-c-cal
> https://github.com/awslabs/aws-c-common
> https://github.com/awslabs/aws-c-compression
> https://github.com/awslabs/aws-c-event-stream
> https://github.com/awslabs/aws-checksums
> https://github.com/awslabs/aws-c-http
> https://github.com/awslabs/aws-c-io
> https://github.com/awslabs/aws-c-mqtt
> https://github.com/awslabs/aws-c-s3
> https://github.com/awslabs/aws-c-sdkutils
> https://github.com/awslabs/aws-lc
> https://github.com/aws/s2n-tls
>
> You will then probably need to modify the setup.py file
> (https://github.com/awslabs/aws-crt-python/blob/v0.16.10/setup.py) so that
> instead of using the cmake build functions have it use the system versions
> of these libraries pre-installed via the buildroot cmake infrastructure.
>
> You can probably use and check for an env variable such as this in the
> setup.py:
> PYTHON_AWSCRT_USE_SYSTEM_CRT=1
>
> When present have it disable building the cmake libraries itself and search
> for the installed system crt libraries instead of using the vendored
> cmake build.
>
> You should upstream this setup.py functionality to the aws-crt-project as
> well
> so that we don't need to carry a patch in buildroot for python-awscrt to
> use
> the system crt libraries.
>
> Then from the python-awscrt buildroot package you would set that env
> variable and add all the cmake subprojects as buildroot package
> dependencies.
>
> First thing you should start with is adding the cmake buildroot packages
> for
> the crt subprojects to buildroot then work on modifying aws-crt-python to
> work correctly with them instead of trying to use its broken cmake
> setuptools
> wrapper.
>
> >
> > Thanks in advance
> >
> > On Thu, 9 Feb 2023 at 10:28, Sourabh Hegde <hrsourabh011 at gmail.com>
> wrote:
> >>
> >> Hi Maxim,
> >>
> >> I was missing python setuptools on my host. After installing it, I can
> add the package. Also, the added package is available in menuconfig
> >>
> >> Thank you for the support!
> >>
> >> On Thu, 9 Feb 2023 at 10:10, Maxim Kochetkov <fido_max at inbox.ru> wrote:
> >>>
> >>>
> >>>
> >>> On 09.02.2023 12:03, Sourabh Hegde wrote:
> >>> > Hello Maxim,
> >>> >
> >>> > Update:
> >>> >
> >>> > The above issue resolved by itself!
> >>> >
> >>> > Now when I do: utils/scanpypi awscrt -o package , it says
> >>> >
> >>> > spdx_lookup module is not installed. This can lead to an inaccurate
> >>> > licence detection. Please install it via
> >>> > pip install spdx_lookup
> >>> > buildroot package name for awscrt: python-awscrt
> >>> > Package: python-awscrt
> >>> > Fetching package awscrt
> >>> > ERROR: Could not find package awscrt.
> >>> > Check syntax inside the python package index:
> >>> > https://pypi.python.org/pypi/ <https://pypi.python.org/pypi/>
> >>> >
> >>> > But the package is available at https://pypi.org/project/awscrt/
> >>> > <https://pypi.org/project/awscrt/>
> >>>
> >>> It works fine for me:
> >>>
> >>> buildroot package name for awscrt: python-awscrt
> >>> Package: python-awscrt
> >>> Fetching package awscrt
> >>> Downloading package awscrt from
> >>>
> https://files.pythonhosted.org/packages/ae/98/dda092078f2d47bad08e184ab9d364b8f89166497062bc1cf811c0bbbde2/awscrt-0.16.10.tar.gz.
> ..
> >>> Checking if package package/python-awscrt already exists...
> >>> Creating package/python-awscrt/python-awscrt.mk...
> >>> More than one file found for license: LICENSE,
> crt/aws-c-common/LICENSE,
> >>> crt/aws-c-io/LICENSE, crt/aws-lc/LICENSE,
> >>> crt/aws-lc/third_party/fiat/LICENSE, crt/aws-c-mqtt/LICENSE,
> >>> crt/aws-checksums/LICENSE, crt/aws-c-s3/LICENSE,
> >>> crt/aws-c-compression/LICENSE, crt/aws-c-http/LICENSE,
> >>> crt/aws-c-sdkutils/LICENSE, crt/s2n/LICENSE, crt/aws-c-cal/LICENSE,
> >>> crt/aws-c-auth/LICENSE, crt/aws-c-event-stream/LICENSE
> >>> Creating package/python-awscrt/python-awscrt.hash...
> >>> Creating package/python-awscrt/Config.in...
> >>> NOTE: Remember to also make an update to the DEVELOPERS file
> >>>        and include an entry for the pkg in packages/Config.in
> >>>
> > _______________________________________________
> > buildroot mailing list
> > buildroot at buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.buildroot.org/pipermail/buildroot/attachments/20230214/dbd3d87a/attachment-0001.html>