[Buildroot] Build error when building two GRUB2 configurations (race condition)

[Stefan Agner]

Hi Yann,

Thanks for your answer!

On 2022-06-25 10:53, Yann E. MORIN wrote:
> Stefan, All,
> 
> On 2022-06-25 09:44 -0700, Stefan Agner spake thusly:
>> In Home Assistant OS we use the capability to build two GRUB2 binaries
>> by enabling these two configurations:
>>
>> BR2_TARGET_GRUB2_I386_EFI=y
>> BR2_TARGET_GRUB2_X86_64_EFI=y
>>
>> Every now and then a from scratch build seems to fail with the following
>> error:
>>
>> config.status: creating config-util.h
>> In file included from ../include/grub/disk.h:***,
>>                  from ../include/grub/file.h:26,
>>                  from ../grub-core/kern/emu/hostfs.c:23:
>> ./config.h:38:10: fatal error: ./config-util.h: No such file or
>> directory
>>    38 | #include <config-util.h>
>>       |          ^~~~~~~~~~~~~~~
>> compilation terminated.
> 
> I had the same issue even with a single grub2 config enabled. This is a
> grub2 issue, and it was supposedly fixed with upstream commit
> 42f4054faf3c (Makefile: Make libgrub.pp depend on config-util.h) and we
> do have this patch backported as:
> boot/grub2/0150-Makefile-Make-libgrub.pp-depend-on-config-util.h.patch

Hm, yeah this seems to be applied properly.

> 
> We currently have grub2 2.04, almost three years old now; we currently
> carry 149 backported patches. This is getting insane. We should update.
> 
> grub 2.06 has been out for a year now, but there are already a lot of
> commits applied in the tree.
> 
> I think we should just bite the bullet and bump to the current HEAD of
> the repository.
> 
> We have (indirect) runtime testing for grub2, so we can at least check
> if bumping is not breaing those (TestIso9660Grub2Hybrid is a two-grub2
> build configuration):
>     support/testing/tests/boot/test_edk2.py
>     support/testing/tests/fs/test_iso9660.py
> 
> New runtime tests to explicitly test grub2, added before we bump, then a
> bump, would be nice. Hint, hint. ;-)
> 
>> At least in this instance it seems to be the second configuration
>> x86_64-efi which fails (as the previous >>> grub2 2.04 Building i386-efi
>> succeeds).
> 
> This is a race condition. As all good race conditions, who wins and who
> lose is totally arbitrary... :-/
> 
>> The full build log can be found here:
>> https://pipelines.actions.githubusercontent.com/serviceHosts/dff1d65b-5367-4f4f-a0ee-c2bf0f874fbd/_apis/pipelines/1/runs/8778/signedlogcontent/14?urlExpires=2022-06-25T16%3A32%3A02.7601449Z&urlSigningMethod=HMACV1&urlSignature=c9ayjKpOOIoTexbMMXYB8A1G6UwmGfhBwTdtxTE3wmI%3D
>>
>> I haven't dig into it really, maybe someone with some familiarity of the
>> GRUB2 (multi-platform) build system has some idea?
> 
> Their Makefiles are, err... interesting. I did have a look a while back
> to try and solve that exact issue, and was relieved to see that they
> supposedly had a patch already. Once I found it was not enough, I did
> not dare look back at the Makefiles again, sadly...


Just run another build, and it had the same outcome. Also this time, it
was the second build, but could be random still I guess.

>From what I can tell config-util.h is required in the all target, and
this is the Makefile part which makes sure it gets created.

config-util.h: stamp-h1
        @test -f $@ || rm -f stamp-h1
        @test -f $@ || $(MAKE) $(AM_MAKEFLAGS) stamp-h1

stamp-h1: $(srcdir)/config-util.h.in $(top_builddir)/config.status
        @rm -f stamp-h1
        cd $(top_builddir) && $(SHELL) ./config.status config-util.h
$(srcdir)/config-util.h.in:  $(am__configure_deps) 
        ($(am__cd) $(top_srcdir) && $(AUTOHEADER))
        rm -f stamp-h1
        touch $@

I wonder if it somehow picks up stamp-h1 from the previous build, or
otherwise races in this section. As this is happening on CI I can't
really inspect the state. I'll try to reproduce locally.

--
Stefan


> Regards,
> Yann E. MORIN.