Can't get template variable substitution to work with profile releases

[fkrause98]

Pre-Check

• If you are filing for a bug, please do a quick search in current issues first
• For questions or support, it helps to include context around your project or problem
• I'm using rebar's latest release and tested this issue with OTP 24 and 25.

Environment

• Add the result of rebar3 report to your message:

$ rebar3 report "as dev1 release"
Rebar3 report
 version 3.18.0
 generated at 2022-05-26T13:46:34+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: as
Entered as:
  as dev1 release
-----------------
Operating System: aarch64-apple-darwin21.4.0
ERTS: Erlang/OTP 25 [erts-13.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]
Root Directory: /Users/fran/.asdf/installs/erlang/25.0
Library directory: /Users/fran/.asdf/installs/erlang/25.0/lib
-----------------
Loaded Applications:
bbmustache: 1.12.2
certifi: 2.8.0
cf: 0.3.1
common_test: 1.23
compiler: 8.2
crypto: 5.1
cth_readable: 1.5.1
dialyzer: 5.0
edoc: 1.2
erlware_commons: 1.5.0
eunit: 2.7.1
eunit_formatters: 0.5.0
getopt: 1.0.1
inets: 8.0
kernel: 8.4
providers: 1.9.0
public_key: 1.13
relx: 4.6.0
sasl: 4.2
snmp: 5.13
ssl_verify_fun: 1.1.6
stdlib: 4.0
syntax_tools: 3.0
tools: 3.5.3

-----------------
Escript path: undefined
Providers:
  app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit get-deps help install install_deps list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade version xref 


...

Current behaviour

❯ DEBUG=1 rebar3 as dev1 release && _build/dev1/rel/rc_example/bin/rc_example
===> Expanded command sequence to be run: [as]
===> Running provider: as
===> Expanded command sequence to be run: [app_discovery,install_deps,lock,compile,release]
===> Running provider: app_discovery
===> Found top-level apps: [rc_example]
	using config: [{src_dirs,["src"]},{lib_dirs,["apps/*","lib/*","."]}]
===> Running provider: install_deps
===> Verifying dependencies...
===> Comparing git ref f15acd3 with f15acd3
===> 25.0 satisfies the requirement for minimum OTP version 22.0
===> Compile (apps)
===> Evaluating config script "/Users/fran/Programming/Erlang/rc_example/_build/default/lib/eleveldb/rebar.config.script"
===> Evaluating config script "/Users/fran/Programming/Erlang/rc_example/_build/default/lib/exometer_core/rebar.config.script"
===> Comparing git ref de9bf01 with de9bf01
===> Comparing git ref fdd3a56 with fdd3a56
===> 25.0 satisfies the requirement for minimum OTP version 22.0
===> Evaluating config script "/Users/fran/Programming/Erlang/rc_example/_build/default/lib/cuttlefish/rebar.config.script"
===> Comparing git ref 013f704 with 013f704
===> 25.0 satisfies the requirement for minimum OTP version 22.0
===> Comparing git ref e293721 with e293721
===> 25.0 satisfies the requirement for minimum OTP version 22.0
===> Compile (apps)
===> Comparing git ref 66c287e with 66c287e
===> Evaluating config script "/Users/fran/Programming/Erlang/rc_example/_build/default/lib/folsom/rebar.config.script"
===> Evaluating config script "/Users/fran/Programming/Erlang/rc_example/_build/default/lib/setup/rebar.config.script"
===> Comparing git ref 07d77f3 with 07d77f3
===> 25.0 satisfies the requirement for minimum OTP version 22.0
===> Comparing git ref 032ea31 with 032ea31
===> Comparing git ref 0c256be with 0c256be
===> 25.0 satisfies the requirement for minimum OTP version 22.0
===> Evaluating config script "/Users/fran/Programming/Erlang/rc_example/_build/default/lib/eleveldb/rebar.config.script"
===> Comparing git ref cca4f1a with cca4f1a
===> 25.0 satisfies the requirement for minimum OTP version 22.0
===> Comparing git ref 7afaad9 with 7afaad9
===> 25.0 satisfies the requirement for minimum OTP version 21.3
===> 25.0 satisfies the requirement for minimum OTP version 21.0
===> Running provider: lock
===> Running provider: compile
===> Compile (apps)
===> Running hooks for compile with configuration:
===> 	{pre_hooks, []}.
===> Compile (project_apps)
===> Running hooks for compile in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{pre_hooks, []}.
===> Running hooks for erlc_compile in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{pre_hooks, []}.
===> Analyzing applications...
===> Compiling rc_example
===> compile options: {erl_opts, [debug_info]}.
===> files to analyze ["/Users/fran/Programming/Erlang/rc_example/src/rc_example_app.erl",
                              "/Users/fran/Programming/Erlang/rc_example/src/rc_example_sup.erl",
                              "/Users/fran/Programming/Erlang/rc_example/src/rc_example.erl",
                              "/Users/fran/Programming/Erlang/rc_example/src/rc_example_vnode.erl"]
===> Running hooks for erlc_compile in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{post_hooks, []}.
===> Running hooks for app_compile in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{pre_hooks, []}.
===> Running hooks for app_compile in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{post_hooks, []}.
===> Running hooks for compile in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{post_hooks, []}.
===> Running hooks for compile with configuration:
===> 	{post_hooks, []}.
===> Running provider: release
===> Configuring releases the {relx, ...} entry from rebar.config
===> Running app-specific hooks
===> Running hooks for release in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{pre_hooks, []}.
===> Running project-wide hooks
===> Running hooks for release with configuration:
===> 	{pre_hooks, []}.
===> Solving Release rc_example-0.1.0
===> Resolved rc_example-0.1.0
===> release: rc_example-0.1.0
     erts: 13.0
     goals:
          rc_example
     applications:
          {kernel,"8.4"}
          {stdlib,"4.0"}
          {compiler,"8.2"}
          {cuttlefish,"3.0.1"}
          {sasl,"4.2"}
          {crypto,"5.1"}
          {riak_sysmon,"2.2.0+build.115.refcca4f1a"}
          {os_mon,"2.7.1"}
          {eleveldb,"riak_kv-3.0.0+build.675.refe293721"}
          {pbkdf2,"2.1.0+build.2686.ref07d77f32"}
          {poolboy,"riak_kv-3.0.0"}
          {hut,"1.3.0"}
          {setup,"2.1.0"}
          {bear,"1.0.0"}
          {folsom,"1.0.0"}
          {syntax_tools,"3.0"}
          {parse_trans,"3.4.1"}
          {exometer_core,"1.6.1"}
          {clique,"0.3.11+build.196.reffdd3a56"}
          {cluster_info,"2.1.0+build.108.ref013f704"}
          {basho_stats,"1.1.0+build.51.refde9bf01"}
          {riak_ensemble,"riak_kv-3.0.0+build.235.ref0c256be",load}
          {riak_core,"riak_kv-3.0.9+build.2065.reff15acd3"}
          {rc_example,"0.1.0"}

===> Assembling release rc_example-0.1.0...
===> Release output dir /Users/fran/Programming/Erlang/rc_example/_build/dev1/rel/rc_example
===> Rewriting .app file: /Users/fran/.asdf/installs/erlang/25.0/lib/kernel-8.4/ebin/kernel.app
===> Rewriting .app file: /Users/fran/.asdf/installs/erlang/25.0/lib/stdlib-4.0/ebin/stdlib.app
===> Rewriting .app file: /Users/fran/.asdf/installs/erlang/25.0/lib/compiler-8.2/ebin/compiler.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/cuttlefish/ebin/cuttlefish.app
===> Rewriting .app file: /Users/fran/.asdf/installs/erlang/25.0/lib/sasl-4.2/ebin/sasl.app
===> Rewriting .app file: /Users/fran/.asdf/installs/erlang/25.0/lib/crypto-5.1/ebin/crypto.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/riak_sysmon/ebin/riak_sysmon.app
===> Rewriting .app file: /Users/fran/.asdf/installs/erlang/25.0/lib/os_mon-2.7.1/ebin/os_mon.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/eleveldb/ebin/eleveldb.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/pbkdf2/ebin/pbkdf2.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/poolboy/ebin/poolboy.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/hut/ebin/hut.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/setup/ebin/setup.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/bear/ebin/bear.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/folsom/ebin/folsom.app
===> Rewriting .app file: /Users/fran/.asdf/installs/erlang/25.0/lib/syntax_tools-3.0/ebin/syntax_tools.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/parse_trans/ebin/parse_trans.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/exometer_core/ebin/exometer_core.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/clique/ebin/clique.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/cluster_info/ebin/cluster_info.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/basho_stats/ebin/basho_stats.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/riak_ensemble/ebin/riak_ensemble.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/default/lib/riak_core/ebin/riak_core.app
===> Rewriting .app file: /Users/fran/Programming/Erlang/rc_example/_build/dev1/lib/rc_example/ebin/rc_example.app
===> release start script created
===> Release successfully assembled: _build/dev1/rel/rc_example
===> Running app-specific hooks
===> Running hooks for release in app rc_example (/Users/fran/Programming/Erlang/rc_example) with configuration:
===> 	{post_hooks, []}.
===> Running project-wide hooks
===> Running hooks for release with configuration:
===> 	{post_hooks, []}.
=INFO REPORT==== 26-May-2022::10:53:17.491046 ===
Invalid node name!
Please check your configuration

=SUPERVISOR REPORT==== 26-May-2022::10:53:17.491064 ===
    supervisor: {local,net_sup}
    errorContext: start_error
    reason: {'EXIT',nodistribution}
    offender: [{pid,undefined},
               {id,net_kernel},
               {mfargs,{net_kernel,start_link,
                                   [#{clean_halt => true,name => '{{node}}',
                                      name_domain => longnames,
                                      supervisor => net_sup}]}},
               {restart_type,permanent},
               {significant,false},
               {shutdown,2000},
               {child_type,worker}]

Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}})

Crash dump is being written to: erl_crash.dump...done

Expected behaviour

I expected the release to be executed, and have as node name "rc_example1@127.0.0.1" as I have stated in my config, seems like the node name is not setting as it should be because of the [#{clean_halt => true,name => '{{node}}'... line in the error output. If I hard code the values, the release works.

This is what I have in my rebar.config:

{relx, [{release, {rc_example, "0.1.0"}, [rc_example]},
        {dev_mode, true},
        {include_erts, false},
        {sys_config, "conf/sys.config"},
        {vm_args, "conf/vm.args"},
        {overlay, [{template, "conf/sys.config", "releases/{{default_release_version}}/sys.config"},
           {template, "conf/vm.args", "releases/{{default_release_version}}/vm.args"}]},
        {extended_start_script, false}]}.

{profiles, [{dev1, [{relx, [{overlay_vars, "conf/vars_dev1.config"}]}]},
            {dev2, [{relx, [{overlay_vars, "conf/vars_dev2.config"}]}]},
            {dev3, [{relx, [{overlay_vars, "conf/vars_dev3.config"}]}]}]}.

And inside each profile file:

%% conf/vars_dev1.config
{node, "rc_example1@127.0.0.1"}.

{web_port,          8198}.
{handoff_port,      8199}.

%% conf/vars_dev2.config
{node, "rc_example2@127.0.0.1"}.

{web_port,          8298}.
{handoff_port,      8299}.

%% conf/vars_dev3.config
{node, "rc_example3@127.0.0.1"}.

{web_port,          8398}.
{handoff_port,      8399}.

This variables are supposed to be used by both the sys.config and vm.args files:

%% conf/sys.config
[{riak_core,
  [{ring_state_dir, "./data/ring"},
   {web_port, {{web_port}}},
   {handoff_port, {{handoff_port}}},
   {schema_dirs, ["lib/rc_example-0.1.0/priv"]}]}].

%% conf/vm.args
-name {{node}}

I've been reading rebar3's documentation and everything looks ok, I can't really figure out/debug why the node's name is not setting as it should be. If it helps, I'm following this (outdated) tutorial https://github.com/lambdaclass/riak_core_tutorial#3-setting-up-the-cluster which was originally written for OTP 19, 20 and 21 and rebar 3.6.1.

[fkrause98]

I have added the non working configuration to this repo, under the linked branch (non_working_profiles) https://github.com/fkrause98/rc_example/tree/non_working_profiles. You should be able to reproduce this error by just cloning, doing git checkout non_working_profiles and running
$ rebar3 as dev1 release && _build/dev1/rel/rc_example/bin/rc_example

You might get this error output:

  format: "Invalid node name!\nPlease check your configuration\n"
    label: {error_logger,info_msg}
2022-05-26 12:14:48.073238 supervisor_report   #{label=>{supervisor,start_error},report=>[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{id,net_kernel},{mfargs,{net_kernel,start_link,[['{{node}}',longnames],true,net_sup]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}
2022-05-26 12:14:48.073302 supervisor_report   #{label=>{supervisor,start_error},report=>[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{id,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}
2022-05-26 12:14:48.073649 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.60.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,401}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}},{ancestors,[net_sup,kernel_sup,<0.47.0>]},{message_queue_len,1},{messages,[{'EXIT',<0.57.0>,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}]},{links,[]},{dictionary,[{longnames,true}]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,28},{reductions,4397}],[]]}
2022-05-26 12:14:48.073678 crash_report        #{label=>{proc_lib,crash},report=>[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.46.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,"application_master.erl"},{line,138}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}},{ancestors,[<0.45.0>]},{message_queue_len,1},{messages,[{'EXIT',<0.47.0>,normal}]},{links,[<0.45.0>,<0.44.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,28},{reductions,193}],[]]}
2022-05-26 12:14:48.076089 std_info            #{label=>{application_controller,exit},report=>[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}
{"Kernel pid terminated",application_controller,"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{ker

Crash dump is being written to: erl_crash.dump...done

But the error is still the same, an invalid node name and inside the error output:
..[['{{node}}',longnames],true,net_sup]}}...

{{node}} instead of its substitution

[ferd]

My guess is that overlays are getting overwritten in a later pass when {sys_config, "conf/sys.config"}, take effects and clobbers the file. You should be able to validate this by having a different file for the overlay and for the sys_config option and seeing which one wins.

My understanding is that the overlays and sys.config files are not expected to be part of the same set, but rather operate as unrelated elements.

[fkrause98]

Sorry, I don't think I follow you quite right, can you go a bit more into detail? 😅

[ferd]

Okay I found the issue and it's a quick fix: #2711

[fkrause98]

Ok, I fixed It locally, for some reason the variable default_release_version is not set, so changing from this:

{relx, [{release, {rc_example, "0.1.0"}, [rc_example]},
        {dev_mode, true},
        {include_erts, false},
        {sys_config, "conf/sys.config"},
        {vm_args, "conf/vm.args"},
        {extended_start_script, false},
        {overlay, [{template, "conf/sys.config", "releases/{{default_release_version}}/sys.config"},
            {template, "conf/vm.args", "releases/{{default_release_version}}/vm.args"}]}]}.

To this

{relx, [{release, {rc_example, "0.1.0"}, [rc_example]},
        {dev_mode, true},
        {include_erts, false},
        {sys_config, "conf/sys.config"},
        {vm_args, "conf/vm.args"},
        {extended_start_script, false},
        {overlay, [{template, "conf/sys.config", "releases/{{release_version}}/sys.config"},
            {template, "conf/vm.args", "releases/{{release_version}}/vm.args"}]}]}.

Fixes my issue, at least