Skip to content

Conversation

@michaelklishin
Copy link
Collaborator

@michaelklishin michaelklishin commented Nov 6, 2025

HEX_TARBALL_FILES included ../../rabbitmq-components.mk which modern rebar3 compile considers to be unsafe and fails on (quite reasonably).

FTR, the error looks like this:

===> Fetching rebar_alias v0.2.0
===> Analyzing applications...
===> Compiling rebar_alias
===> Verifying dependencies...
===> Fetching amqp_client v4.1.5
escript: exception error: no function clause matching
                 rebar_fetch:format_error({error,
                                           {hex_tarball,
                                            {inner_tarball,
                                             {"../../rabbitmq-components.mk",
                                              unsafe_path}}}}) (/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_fetch.erl, line 99)
  in function  rebar3:handle_error/2 (/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar3.erl, line 383)
  in call from escript:run/2 (escript.erl, line 904)
  in call from escript:start/1 (escript.erl, line 418)
  in call from init:start_it/1
  in call from init:start_em/1
  in call from init:do_boot/3

Excluding this relative path addresses the issue.

rabbit_common/Makefile arguably does not need to
add any .mk files to the tarball but let's
deal with that in a separate change.

HEX_TARBALL_FILES included ../../rabbitmq-components.mk which
modern `rebar3 compile` considers to be unsafe and fails on
(quite reasonably).

FTR, the error looks like this:

````
===> Fetching rebar_alias v0.2.0
===> Analyzing applications...
===> Compiling rebar_alias
===> Verifying dependencies...
===> Fetching amqp_client v4.1.5
escript: exception error: no function clause matching
                 rebar_fetch:format_error({error,
                                           {hex_tarball,
                                            {inner_tarball,
                                             {"../../rabbitmq-components.mk",
                                              unsafe_path}}}}) (/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar_fetch.erl, line 99)
  in function  rebar3:handle_error/2 (/home/runner/work/rebar3/rebar3/apps/rebar/src/rebar3.erl, line 383)
  in call from escript:run/2 (escript.erl, line 904)
  in call from escript:start/1 (escript.erl, line 418)
  in call from init:start_it/1
  in call from init:start_em/1
  in call from init:do_boot/3
```

Excluding this relative path addresses the issue.

rabbit_common/Makefile arguably does not need to
add any .mk files to the tarball but let's
deal with that in a separate change.
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR removes the rabbitmq-components.mk file from the Hex.pm tarball configuration across multiple RabbitMQ dependency packages.

  • Excludes ../../rabbitmq-components.mk from HEX_TARBALL_FILES in four Makefiles
  • Ensures only package-specific files (like git-revisions.txt and mk files) are included in Hex.pm tarballs
  • The build infrastructure file from the parent directory is still included for build purposes

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
deps/rabbit_common/Makefile Removes ../../rabbitmq-components.mk from HEX_TARBALL_FILES, retaining only git-revisions.txt and mk/ directory files
deps/amqp_client/Makefile Removes ../../rabbitmq-components.mk from HEX_TARBALL_FILES, leaving only git-revisions.txt
deps/amqp10_common/Makefile Removes ../../rabbitmq-components.mk from HEX_TARBALL_FILES, leaving only git-revisions.txt
deps/amqp10_client/Makefile Removes ../../rabbitmq-components.mk from HEX_TARBALL_FILES, leaving only git-revisions.txt

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@michaelklishin michaelklishin changed the title Fix rabbit_common, amqp_client, amqp10_client publishing to hex.pm Build system: fix rabbit_common, amqp_client, amqp10_client publishing to hex.pm Nov 6, 2025
@michaelklishin michaelklishin added this to the 4.3.0 milestone Nov 6, 2025
@dumbbell
Copy link
Collaborator

dumbbell commented Nov 7, 2025

The rabbitmq-components.mk file is required to make sure developers of plugins use the same dependencies as RabbitMQ.

I understand why we can’t include a file from a parent directly, but I don’t get why the tarball seems incorrectly created. I will need to look at what is created.

@lukebakken
Copy link
Collaborator

lukebakken commented Nov 7, 2025

@michaelklishin when I run make -C deps/amqp_client hex-tarball-create I get this error:

 TAR    .erlang.mk/amqp_client.tar
Error trying to open file "git-revisions.txt": enoent

The only way I've found to workaround that error is to run the following commands prior:

make source-dist
$ cp -v PACKAGES/rabbitmq-server-4.2.0+beta.4.110.g34c6ef4/git-revisions.txt deps/amqp_client/
'PACKAGES/rabbitmq-server-4.2.0+beta.4.110.g34c6ef4/git-revisions.txt' -> 'deps/amqp_client/git-revisions.txt'

That can't possibly be right eh?

Anyway, when I work around that issue, and create the tarball, here are the contents:

$ tar tf .erlang.mk/amqp_client.tar
VERSION
CHECKSUM
metadata.config
contents.tar.gz

$ tar -C /tmp -xf .erlang.mk/amqp_client.tar

$ tar tf /tmp/contents.tar.gz
ebin/amqp_client.app
include/amqp_client.hrl
include/amqp_client_internal.hrl
include/amqp_gen_consumer_spec.hrl
include/rabbit_routing_prefixes.hrl
LICENSE
LICENSE-MPL-RabbitMQ
Makefile
README.in
src/amqp_auth_mechanisms.erl
...
...
...
src/amqp_util.erl
src/overview.edoc.in
src/rabbit_routing_util.erl
src/uri_parser.erl
tar: Removing leading `../../' from member names
../../rabbitmq-components.mk
git-revisions.txt

@dumbbell what happens is that rabbitmq-components.mk is added with relative paths, which I'm assuming rebar3 doesn't like. We used to have symlinks to deal with this issue but they were removed in #14081.

Since compiling RabbitMQ on Windows isn't really supported, maybe the symlinks should be restored. Or, I'm just about to investigate a change here that would strip all leading ../ from file paths used as metadata in the hex tar file.

UPDATE:
FWIW, this patch does remove all leading ../ from filenames in the output tarball -
erlang.mk.patch

@lukebakken
Copy link
Collaborator

@michaelklishin I created this project to see what would happen if I included the latest amqp_client in mix-based and rebar3-based projects.

In the mix directory, when I run mix deps.get && mix compile, everything builds correctly. It seems that mix is smart enough to create the appropriate rebar.config files for each dependency:

$ cd mix
$ mix deps.get && mix compile
$ cat deps/amqp_client/rebar.config
{deps, [
{rabbit_common,"4.2.1"},{credentials_obfuscation,"3.5.0"}
]}.
{erl_opts, [debug_info,warn_export_vars,warn_shadow_vars,warn_obsolete_guard,deterministic]}.

There's also a simple rebar3 project that works correctly as well. I'm using this version of rebar3:

$ rebar3 version
rebar 3.25.1 on Erlang/OTP 28 Erts 16.1.1

It's the latest version - maybe this handles leading ../ in tarballs correctly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants