Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libelf deprecation on macOS #1653

Open
iMichka opened this issue Feb 1, 2024 · 26 comments
Open

libelf deprecation on macOS #1653

iMichka opened this issue Feb 1, 2024 · 26 comments
Labels
enhancement New feature or request

Comments

@iMichka
Copy link

iMichka commented Feb 1, 2024

Hello

I am a maintainer of the Homebrew project and wanted to give a small heads up: we would like to deprecate libelf on macOS.

The upstream libelf website is gone, the remaining source code can only be through archive.org ... (since at least 2019, which was 5 years ago ...). avrdude is one of the last projects that is using libelf in macOS.

Are there any plans for a replacement solution on your side?

@mcuee mcuee added the enhancement New feature or request label Feb 2, 2024
@mcuee
Copy link
Collaborator

mcuee commented Feb 2, 2024

@iMichka

What is the replacement library recommended by Homebrew project?

@mcuee
Copy link
Collaborator

mcuee commented Feb 2, 2024

As of now, I do not know of any plans to use an alternative library for the avrdude project.

BTW, for Windows MSVC build, we have a local version of libelf.
https://github.com/avrdudes/libelf

@mcuee
Copy link
Collaborator

mcuee commented Feb 2, 2024

Google says a potential "replacement" of libelf is elfutils. But then Homebrew says it is Linux only.
https://formulae.brew.sh/formula/elfutils
https://github.com/Homebrew/homebrew-core/blob/1cde8401d8be5f28d74a402afe67fd52ac0575ed/Formula/e/elfutils.rb

@mcuee
Copy link
Collaborator

mcuee commented Feb 2, 2024

Windows MSVC build: local fork version of the legacy libelf.
https://github.com/avrdudes/libelf

Windows: MSYS2 mingw uses legacy libelf.
https://packages.msys2.org/package/mingw-w64-x86_64-libelf

macOS: Macports uses legacy libelf
https://ports.macports.org/port/libelf/details/

macOS: Homebrew still uses legacy libelf now but mark as deprecated.
https://formulae.brew.sh/formula/libelf

Debian Linux -- using elfutils to provide libelf
https://packages.debian.org/search?keywords=libelf

Arch Linux -- using elfutils to provide libelf
https://archlinux.org/packages/core/x86_64/libelf/

FreeBSD/OpenBSD -- using elftoolchain to provide libelf
https://wiki.freebsd.org/LibElf
https://man.openbsd.org/elf
https://sourceforge.net/p/elftoolchain/wiki/Home/

NetBSD seems to be still using legacy libelf.
https://ftp.netbsd.org/pub/NetBSD/NetBSD-current/pkgsrc/devel/libelf/index.html
https://pkgsrc.se/devel/libelf

@mcuee
Copy link
Collaborator

mcuee commented Feb 2, 2024

Looks like there is no easy replacement of libelf for macOS now. So I think this may not be an easy task and we may not be able to deal with it anytime soon.

@dl8dtl
Copy link
Contributor

dl8dtl commented Feb 2, 2024

Macports have a port of elftoolchain.
FreeBSD has a port of elfutils, so it's not quite clear why it is said to be "Linux only".
Other than this: the usual reason for deprecating something is that there's a better replacement available, rather than something not being developed anymore. There are many things that are no longer being developed since they are fairly complete, and just don't need further development. Take TaylorUUCP for an example: every MacOS these days still comes with it. ;-)

@mcuee
Copy link
Collaborator

mcuee commented Feb 2, 2024

FreeBSD has a port of elfutils, so it's not quite clear why it is said to be "Linux only".

I think it is from Homebrew project point of view. Homebrew supports macOS (normal Homebrew) and Linux. As of now, it seems Homebrew only supports elfutils under Linux and not macOS.

@mcuee
Copy link
Collaborator

mcuee commented Feb 2, 2024

Macports have a port of elftoolchain.

We need to check if it works for the purpose of avrdude though. If it works, then the solution is to ask Homebrew to provide elftoolchain.

@dl8dtl
Copy link
Contributor

dl8dtl commented Feb 2, 2024

Macports have a port of elftoolchain.

We need to check if it works for the purpose of avrdude though.

Under FreeBSD, the (OS-provided) elftoolchain does the job. I no longer reference any 3rd-party ELF library in the devel/avrdude port.

Either way, my main point was, don't deprecate something unless there's a replacement which is at least as good as the deprecated item. FreeBSD still also offers the old libelf port (rather than kicking it out), but of course, installing it would conflict with elfutils then. You can only install one of those, because they install files into the same place.

@stefanrueger
Copy link
Collaborator

Thanks for the advance warning @iMichka. I agree with @dl8dtl that it seems unfortunate to deprecate elflib without an obvious replacement. AVR elf files won't have changed in the last decade or so, so that a library reading them doesn't need to be "new".

@mcuee @MCUdude Is is possible to drop the reliance of MacOS on libelf before the new AVRDUDE release next week?

@mcuee
Copy link
Collaborator

mcuee commented Feb 3, 2024

@stefanrueger

I think we should stick to libelf on macOS with avrdude 7.3 release.

Then we can explore the alternative for avrdude 7.4 release.

@mcuee
Copy link
Collaborator

mcuee commented Feb 3, 2024

Macports have a port of elftoolchain.

We need to check if it works for the purpose of avrdude though. If it works, then the solution is to ask Homebrew to provide elftoolchain.

I just took a look at elftoolchain macports and it is not in a good shape -- failed under Apple Silicon and almost no users (2 downloads in 30 days).
https://ports.macports.org/port/elftoolchain/details/

macports libelf is quite healthy (no issues with Apple Silicon, 641 downloads in 30 days)
https://ports.macports.org/port/libelf/details/

I think we can rule out the use of elftoolchain provided libelf as a candidate to replace libelf.

@mcuee
Copy link
Collaborator

mcuee commented Feb 3, 2024

As for elfutils, I do not see much efforts to port it to macOS. Google only finds one with the last commit date on 2019-Dec-24.
https://github.com/kdab-hotspot/elfutils

Unfortunately it does not seem to build well under my Mac Mini M1 (Apple macOS 14.3 with fully updated Homebrew). Good thing is that the sub libelf source codes seem to produce a static link library libelf.a.

mcuee@mcuees-Mac-mini build % pwd
/Users/mcuee/build/avr/avrdude_test/elfutils_macos/build

mcuee@mcuees-Mac-mini build %
CC=gcc-13 CFLAGS="-I/opt/homebrew/include -I/opt/homebrew/opt/gettext/include/" LDFLAGS="-L/opt/homebrew/opt/argp-standalone/lib/" PKG_CONFIG_PATH=/opt/homebrew/opt/libarchive/lib/pkgconfig ../configure --enable-maintainer-mode --disable-symbol-versioning

mcuee@mcuees-Mac-mini build % make
...
...
Making all in libcpu
  CC       i386_gendis.o
../../libcpu/i386_gendis.c: In function 'main':
../../libcpu/i386_gendis.c:70:10: error: 'error_message_count' undeclared (first use in this function)
   70 |   return error_message_count != 0;
      |          ^~~~~~~~~~~~~~~~~~~
../../libcpu/i386_gendis.c:70:10: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [i386_gendis.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

mcuee@mcuees-Mac-mini build % make -i
...
...
all kinds of errors
...
...
mcuee@mcuees-Mac-mini build % cd libelf 

mcuee@mcuees-Mac-mini libelf % ls -la libelf.a
-rw-r--r--  1 mcuee  staff  270096 Feb  3 13:05 libelf.a

mcuee@mcuees-Mac-mini libelf % file libelf.a
libelf.a: current ar archive

@iMichka
Copy link
Author

iMichka commented Feb 3, 2024

What is the replacement library recommended by Homebrew project?

It's a complex situation. I have read all your comments, thanks for investigating.
One solution is maybe binutils that might bring equivalent tooling. It depends on what you were doing with libel.

I think we should stick to libelf on macOS with avrdude 7.3 release.
Then we can explore the alternative for avrdude 7.4 release.

That's fine, don't worry, there is no rush here.

elfutils

elfutils needs massive patching to be able to build it on macOS, so that's probably not the first solution I would consider.

@mcuee mcuee added this to the AVRDUDE 7.4 milestone Feb 6, 2024
@mcuee
Copy link
Collaborator

mcuee commented Feb 7, 2024

There is of course another option to bundle libelf with avrdude.

For example, we may use the following libelf for Windows and make it compatible with macOS.
https://github.com/avrdudes/libelf

@MCUdude
Copy link
Collaborator

MCUdude commented Feb 8, 2024

@iMichka I have brew/avrdude related question. Avrdude 7.3 has a new dependency; libserialport. It's being used for automatic serial port discovery. However, Brew doesn't build Avrdude with libserialport. How can we change this?

% avrdude -curclock -patmega328p -Pch340
avrdude error: avrdude built without libserialport support; please compile again with libserialport installed
avrdude warning: serial adapter ch340 not connected to host
avrdude OS error: cannot open port ch340: No such file or directory
avrdude error: unable to open port ch340 for programmer urclock
avrdude error: avrdude built without libserialport support; please compile again with libserialport installed

avrdude done.  Thank you.

@mcuee
Copy link
Collaborator

mcuee commented Feb 9, 2024

@MCUdude

The forumla needs to be updated -- it is missing libserialport dependancy.

  #need to add the following line
  depends_on "libserialport"

Reference:
https://github.com/Homebrew/homebrew-core/blob/ca4f6490a40087a4789c8f8e934a21ed3a3bc233/Formula/a/avrdude.rb

  depends_on "cmake" => :build
  depends_on "hidapi"
  depends_on "libftdi"
  depends_on "libusb"
  depends_on "libusb-compat"
  #need to add the following line
  depends_on "libserialport"

  uses_from_macos "bison" => :build
  uses_from_macos "flex" => :build

  on_macos do
    # https://github.com/avrdudes/avrdude/issues/1653
    depends_on "libelf" => :build
  end

  on_linux do
    depends_on "elfutils"
    depends_on "readline"
  end

You may want to raise an issue here.
https://github.com/Homebrew/homebrew-core/issues

BTW, I usually use HEAD (to build from latest git main source codes) so I did not notice this issue.

brew install avrdude
brew unlink avrdude
brew install avrdude --HEAD         (to following avrdude git main)
brew reinstall avrdude           (to catch up with latest git main)

@mcuee
Copy link
Collaborator

mcuee commented Feb 9, 2024

Link to Homebrew issue here.

libelf - has two dependents avrdude and dynamips. We've already migrated both of these to elfutils on Linux but we can't do so on macOS because elfutils is Linux-only.

One comment in the above issue discussion.

My libelf migration plan was elftoolchain (Homebrew/homebrew-core#104303) though not sure of actual compatibility.

But apparently the above efforts to get elftoolchain into Homebrew failed.

@mcuee
Copy link
Collaborator

mcuee commented Feb 24, 2024

From the following discussion.

Dynamips already points users to elfutils-libelf-devel on Redhat based systems, so it's already a compatible replacement.
Why not have a formula for elfutils-libelf in Homebrew?

The whole of elfutils might be hard due to linux-only stuff, but the libelf in elfutils by itself should be doable.
Then Homebrew can deprecate libelf in favor of elfutils-libelf.

@mcuee
Copy link
Collaborator

mcuee commented Feb 28, 2024

@MCUdude

Do you have the right for the following repo? If yes, please add me there as well. Thanks.
https://github.com/avrdudes/libelf

@dl8dtl
Copy link
Contributor

dl8dtl commented Feb 28, 2024

@MCUdude

Do you have the right for the following repo? If yes, please add me there as well. Thanks. https://github.com/avrdudes/libelf

I added you all.

@mcuee
Copy link
Collaborator

mcuee commented Mar 2, 2024

@mariusgreuel has done a good job that avrdude-libelf actually builds under macOS. In fact it builds on Windows, Linux and macOS with CMake.

@mcuee mcuee removed this from the AVRDUDE 7.4 milestone Mar 17, 2024
@mcuee
Copy link
Collaborator

mcuee commented Mar 17, 2024

Remove this from avrdude 7.4 milestone since there is no active development on this topic.

@flaviojs
Copy link

flaviojs commented May 4, 2024

If the only issue is having source code available, then this page looks like a good new reference:
https://fossies.org/linux/misc/old/libelf-0.8.13.tar.gz/

@stefanrueger
Copy link
Collaborator

Sooo, we are planning to release AVRDUDE 8.0 sometime soon-ish. Does anyone (ideally with a Mac!) have a plan how to deal with the libelf deprecation on macOS? @MCUdude @mcuee is it a case of installing libelf source for the Mac and linking to it statically? Just to be clear I won't be able to do that but is anyone else?

@mcuee
Copy link
Collaborator

mcuee commented Aug 2, 2024

Sooo, we are planning to release AVRDUDE 8.0 sometime soon-ish. Does anyone (ideally with a Mac!) have a plan how to deal with the libelf deprecation on macOS? @MCUdude @mcuee is it a case of installing libelf source for the Mac and linking to it statically? Just to be clear I won't be able to do that but is anyone else?

My take is to close this issue as unplanned.

Worst case is just to change our CMake script to use our own libelf repo.
avrdudes/libelf#1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

6 participants