Arduino Crossbuild container and avrdude-packing script improvement

[mcuee]

From #1594 (comment)
@stefanrueger and @MCUdude

One possibility to sort out the old Arduino Crossbuild MinGW cross compiler issue (it uses the default mingw64 cross compiler for i686), is to give up on the MinGW cross build (we can easily remove it from github action). We anyway recommend people to use the MSVC build first and then MinGW build if the MSVC build does not work. The original intention is anyway to provide static linked version for Linux and macOS users.

For the use case of Arduino Core, I think the MSVC build should work pretty well since the limitations do not normally affect Arduino Core users. Or we can use the MinGW build as well.

Reference:
https://github.com/avrdudes/avrdude/wiki/Getting-Nightly-Builds-for-AVRDUDE

[mcuee]

From #1594 (comment) by @stefanrueger.

@umbynos I understand that the Arduino build process either does not compile with at least C99 compatibility or does not link to libraries that are C99-compliant. This has caused problems and may harbour other (so far unknown) problems, because the AVRDUDE project has been relying on availability of C99 (which is 23 years old)

Can the Arduino-packaging be made to compile with C99 and link to C99 standard C libraries?

[mcuee]

From here: to add libserialport support

• Add libserialport for future avrdude release arduino/crossbuild#13

The other solution is to build libserialport within the following github action.
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing.yml

[mcuee]

Reference: https://github.com/avrdudes/avrdude/wiki/Getting-Nightly-Builds-for-AVRDUDE

As of now, avrdude project provides three types of Windows binaries from github action.

1. MSVC build. This is the version we recommend the user to use under Windows (mainly the x64 version)

MSVC build (using VS2022, x86, x64 and arm64 build), without libserialport, using avrdude's own libusb/libftdi, using avrdude's own readline wrapper, using hidapi 0.13.0 release
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/build.yml
https://github.com/avrdudes/avrdude/wiki/Getting-Nightly-Builds-for-AVRDUDE#windows-static-linking-to-dependancies (latest run for git main)

Known Issues for this version:

• To add libserialport support to Windows MSVC build #1552
• ftdi_syncbb: very slow D2XX based MSVC build compared to libftdi based MinGW build #1253
• avrdude MSVC build problem with USB composite device (avrdude-libusb library issue) -- low priority #968
• CH341A futher improvement -- to support MSVC build #1440

2. MSYS2 mingw32/mingw64 build
   This is the second choice for Windows users (mainly mingw64 version)

MSYS2 mingw32/64 (x86 and x64 build), with libserialport, with GNU Readline, with libusb-1.0.26 and libusb-compat-0.1, with libftdi1, with hidapi 0.14.0.
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/build.yml
https://github.com/avrdudes/avrdude/wiki/Getting-Nightly-Builds-for-AVRDUDE#windows-static-linking-to-dependancies (latest run for git main)

Known limitation of this build

• To adopt PR #1264 approach of using Readline replacement for MinGW build (Low Priority) #1271 (timeout may be more likely to happen for -c urclock in terminal mode)

3. Arduino avrdude-packing 32bit Windows binary
   This is likely to be used by Arduino distributions under Windows, Linux and macOS.

using Arduino avrdude-packing script (32bit x86 build only), using a Linux container with cross compiler, without libserialport, with libusb-1.0.26 and libusb-compat-0.1.7, libftdi1-1.4, hidapi 0.12.0, GNU Readline 8.0.
https://github.com/avrdudes/avrdude/blob/main/.github/workflows/arduino_packing.yml
https://github.com/avrdudes/avrdude/wiki/Getting-Nightly-Builds-for-AVRDUDE#linux-macos-and-windows-static-linking-to-dependancies (latets run for git main)

Known limitation of this build

• To adopt PR #1264 approach of using Readline replacement for MinGW build (Low Priority) #1271 (timeout may be more likely to happen for -c urclock in terminal mode)
• Add libserialport for future avrdude release arduino/crossbuild#13 (no libserialport support)

[mcuee]

Reference:
https://www.msys2.org/docs/environments/

MSVCRT vs UCRT
These are two variants of the C standard library on Microsoft Windows.

**MSVCRT (Microsoft Visual C++ Runtime) is available by default on all Microsoft Windows versions, but due to backwards compatibility issues is stuck in the past, not C99 compatible and is missing some features.

• It isn't C99 compatible, for example the printf() function family, but...
• mingw-w64 provides replacement functions to make things C99 compatible in many cases
• It doesn't support the UTF-8 locale
• Binaries linked with MSVCRT should not be mixed with UCRT ones because the internal structures and data types are different. (More strictly, object files or static libraries built for different targets shouldn't be mixed. DLLs built for different CRTs can be mixed as long as they don't share CRT objects, e.g. FILE*, across DLL boundaries.) Same rule is applied for MSVC compiled binaries because MSVC uses UCRT by default (if not changed).
• Works out of the box on every version of Microsoft Windows.

UCRT (Universal C Runtime) is a newer version which is also used by Microsoft Visual Studio by default. It should work and behave as if the code was compiled with MSVC.

• Better compatibility with MSVC, both at build time and at run time.
• It only ships by default on Windows 10 and for older versions you have to provide it yourself or depend on the user having it installed.

[mcuee]

Take note UCRT can be installed on Windows 7/8/8.1 as well.
https://support.microsoft.com/en-us/topic/update-for-universal-c-runtime-in-windows-c0514201-7fe6-95a3-b0a5-287930f3560c

[mcuee]

@stefanrueger

From the above link we can see that msvcrt is not C99 compliant but I am not so sure why only the Arduino cross-compiler has the issue and only under Windows 7.

[mcuee]

@umbynos

The avrdude-packing script in this repo is a cut-down version of your more detailed script. One missing feature is to deal with the tagged release (say 7.3 release). Just wondering if you can help here. Thanks.

[mcuee]

More discussions here:
#1679

I will close this one now.