Compiling with Elixir fails [JIRA: RIAK-3371]

[Lazarus404]

I've tried compiling this on Ubuntu using Elixir 1.3.2 and Elixir 1.4.1/2, which uses Rebar3 under the hood, and it fails. I get in my rebar crashdump:

===> 19.2.3 satisfies the requirement for minimum OTP version R16
===> Expanded command sequence to be run: []
===> Expanded command sequence to be run: [{default,app_discovery},
                                           {bare,compile}]
===> 19.2.3 satisfies the requirement for minimum OTP version R16
===> Compiling cuttlefish
===> run_hooks("/home/lee/Documents/file_server/deps/cuttlefish", pre_hooks, compile) -> no hooks defined

===> run_hooks("/home/lee/Documents/file_server/deps/cuttlefish", pre_hooks, erlc_compile) -> no hooks defined

===> erlopts [debug_info,debug_info,
              {parse_transform,lager_transform},
              warn_untyped_record,warnings_as_errors]
===> files to compile ["/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_generator.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_conf.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_vmargs.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/conf_parse.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_escript.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_rebar_plugin.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_bytesize.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_duration.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_duration_parse.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/lager_stderr_backend.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_flag.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_util.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_effective.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_advanced.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_datatypes.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_enum.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_schema.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_variable.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_translation.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_unit.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_validator.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_error.erl",
                       "/home/lee/Documents/file_server/deps/cuttlefish/src/cuttlefish_mapping.erl"]
===> run_hooks("/home/lee/Documents/file_server/deps/cuttlefish", post_hooks, erlc_compile) -> no hooks defined

===> run_hooks("/home/lee/Documents/file_server/deps/cuttlefish", pre_hooks, app_compile) -> no hooks defined

===> run_hooks("/home/lee/Documents/file_server/deps/cuttlefish", post_hooks, app_compile) -> no hooks defined

===> run_hooks("/home/lee/Documents/file_server/deps/cuttlefish", pre_hooks, escriptize) -> no hooks defined

===> run_hooks("/home/lee/Documents/file_server/deps/cuttlefish", pre_hooks, escriptize) -> no hooks defined

===> Building escript...
===> Creating escript file /home/lee/Documents/file_server/deps/cuttlefish/_build/default/bin/cuttlefish
===> processing <<"cuttlefish">>
===> new deps of <<"cuttlefish">> found to be []
===> processing <<"getopt">>
===> Uncaught error in rebar_core. Run with DEBUG=1 to see stacktrace or consult rebar3.crashdump
===> Uncaught error: {badmatch,error}
===> Stack trace to the error location:
[{rebar_prv_escriptize,find_deps_of_deps,3,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,240}]},
 {rebar_prv_escriptize,find_deps,2,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,234}]},
 {rebar_prv_escriptize,escriptize,2,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,104}]},
 {rebar_prv_escriptize,do,1,
                       [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_escriptize.erl"},
                        {line,80}]},
 {rebar_core,do,2,
             [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_core.erl"},
              {line,125}]},
 {rebar_hooks,run_provider_hooks_,6,
              [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_hooks.erl"},
               {line,49}]},
 {rebar_hooks,run_all_hooks,6,
              [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_hooks.erl"},
               {line,16}]},
 {rebar_prv_compile,compile,3,
                    [{file,"/home/tristan/Devel/rebar3/_build/default/lib/rebar/src/rebar_prv_compile.erl"},
                     {line,129}]}]
===> When submitting a bug report, please include the output of `rebar3 report "your command"`
** (Mix) Could not compile dependency :cuttlefish, "/home/lee/.mix/rebar3 bare compile --paths "/home/lee/Documents/file_server/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile cuttlefish", update it with "mix deps.update cuttlefish" or clean it with "mix deps.clean cuttlefish"

I've posted on the Elixir forum and another dev has stated they get the same result. I need this compiling so I can make use of Riak Core, so it's kinda frustrating. Is anyone aware of why this happens?

Thanks

[mindreader]

I've tried rebar 3.3.6. I get the exact same error. I have no idea how to even begin to debug this. Since I don't know what else to try I'm going to just have to pass on erlang for some other solution.

[dplummer]

I'm at the same place. A few days ago I was able to get riak_core_ng to compile with cuttlefish by using erlang 18.3.4, elixir 1.4.2, rebar3 3.3.2, and on macOS Sierra. However, I cannot reproduce that today, and I have yet to get it to compile in an ubuntu docker container.

You might try with erlang 18, that might give you better results.

[mindreader]

I think I already tried it with erlang 18. I really don't know what it is. I'm very new to erlang and elixir, but from some other issues floating around it sounds like it has something to do with using rebar outside of the cuttlefish package maybe? But mix literally forces you to install rebar3 before it starts so I'm not sure how you'd fix that.

[tslater]

We've been seeing the same issues, so I've been watching this one. I'm not sure where to start looking either.

[dplummer]

I'm trying now to use the rebar3 branch of cuttlefish, but now I'm getting an error on the pre compile neotoma hook:

===> Fetching rebar3_neotoma_plugin ({git,
                                      "https://github.com/basho/rebar3_neotoma_plugin.git",
                                      {branch,"master"}})
===> Fetching neotoma ({git,"https://github.com/basho/neotoma.git",
                            {branch,"feature/riak-2903/rebar3"}})
===> Compiling neotoma
===> Compiling rebar3_neotoma_plugin
===> Compiling cuttlefish
===> Unable to run pre hooks for 'compile', command 'neotoma' not found.
** (Mix) Could not compile dependency :cuttlefish, "/root/.mix/rebar3 bare compile --paths "/src/dist_cache/_build/dev/lib/*/ebin"" command failed. You can recompile this dependency with "mix deps.compile cuttlefish", update it with "mix deps.update cuttlefish" or clean it with "mix deps.clean cuttlefish"
The command '/bin/sh -c mix deps.compile' returned a non-zero code: 1

It should be reproducible, the code is at dplummer/exp_elixir_riak_core#d36ca3b, running docker build .

[SimonWoolf]

For anyone else having this, you can get it to work (with riak_core_ng 3.0.4) by specifying the following dependency overrides in mix.exs:

{:cuttlefish, github: "basho/cuttlefish", tag: "2.0.11", manager: :rebar3, override: true},
{:lager, github: "basho/lager", tag: "3.2.4", manager: :rebar3, override: true},
{:goldrush, github: "basho/goldrush", tag: "0.1.9", manager: :rebar3, override: true}

c.f. Kyorai/riak_core#20 (comment)

[Iulian7]

I still have this problem (with riak_core_ng 3.0.5). I added the relevant overrides but I get this:

===> Compiling src/cuttlefish_unit.erl failed
src/cuttlefish_unit.erl:4: export_all flag enabled - all functions will be exported

===> Compilation failed: {error,[],
                                [["src/cuttlefish_unit.erl:4: export_all flag enabled - all functions will be exported\n"]]}
** (Mix) Could not compile dependency :cuttlefish, "/Users/iulian/.mix/rebar3 bare compile --paths "/Users/iulian/elixir/riak/_build/dev/lib/*/ebin"" command failed. You can
recompile this dependency with "mix deps.compile cuttlefish", update it with "mix deps.update cuttlefish" or clean it with "mix deps.clean cuttlefish"

I am running macOS Sierra, rebar 3.4.1 on Erlang/OTP 20 Erts 9.0 and Elixir 1.4.5.

[queer]

For anyone still having the compilation issues with riak_core_ng 3.0.9, I managed to solve this on Elixir 1.5.2 / OTP 20 by changing the dependency override to

{:cuttlefish, github: "Kyorai/cuttlefish", ref: "649cda2828ef939126a20faa5d0ee0a2ab69f9a6", override: true},

and commenting out the escript-related parts of the cuttlefish rebar.config:

%%{escript_emu_args, "%%! -escript main cuttlefish_escript +S 1 +A 0\n"}.
%%{escript_incl_apps, [goldrush, getopt, lager, cuttlefish]}.
%%{escript_main_app, cuttlefish}.

Obviously this is not a great fix for the issue, but at least it works and doesn't seem to cause any issues.