Cannot get conform to apply settings from app.conf at app start about conform

[bitwalker]

Could you describe your deployment workflow for me, just so we're on the same page?

Conform itself isn't responsible for copying the config, that's actually up to the boot script in exrm. However the config is copied from the release directory to running-config during upgrade, downgrade, console, foreground, and start, so if you are copying the config manually after one of those commands are executed, then that would explain why the config used is the old one.

Once I know more about your workflow, I'll try and reproduce locally.

[localshred]

After reading some more code I realized that I was constraining conform and exrm and conform_exrm to dev/test only but that doesn't work when trying to build a prod release, so I removed those restrictions. The conf file is now being copied over correctly and the conform script is now being built into my release. I am now back to the issue I encountered with the old release, namely when the app.sh script tries to generate the config it is failing on this line:

# ...
result="$("$BINDIR/escript" "$__conform" --conf "$__conform_file" --schema "$__schema_file" --config "$SYS_CONFIG" --output-dir "$GENERATED_CONFIG_DIR")"
# ...

Which, expanded out is:

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform --conf /app/gretel/current/running-config/gretel.conf --schema /app/gretel/current/releases/0.0.1/gretel.schema.exs --config /app/gretel/current/running-config/sys.config --output-dir /app/gretel/current/running-config

When I run that command I get the following error that I was getting before:

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform --conf /app/gretel/current/running-config/gretel.conf --schema /app/gretel/current/releases/0.0.1/gretel.schema.exs --config /app/gretel/current/running-config/sys.config --output-dir /app/gretel/current/running-config
escript: exception error: undefined function conform_escript:main/1
  in function  escript:run/2 (escript.erl, line 757)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_it/1
  in call from init:start_em/1

Hoping that provides more insight.

[bitwalker]

That's a new one, does the conform binary actually exist at /app/gretel/current/releases/0.0.1/conform?

[localshred]

Sure, my current deploy process is to build and run on the same machine (for now). The config/gretel.conf file is already on this machine and in the config directory. /app/gretel/current/bin is on the $PATH as well.

1. Pull latest commits on build server
2. MIX_ENV=prod mix do compile, release
3. mkdir /app/gretel/releases/{version} && cd /app/gretel/releases/{version}
4. tar -xvf {path to gretel code}/rel/gretel/releases/{version}/gretel.tar.gz
5. rm -f /app/gretel/current && ln -sF /app/gretel/releases/{version}/ /app/gretel/current
6. gretel console

[localshred]

The conform binary is only present when all three deps are available for the env I'm building for.

[bitwalker]

That's expected, let me review your process and see if I can reproduce, should just be a few minutes.

[localshred]

Ok thx!

[localshred]

My app deps, if it matters:

  defp deps do
    [
      { :conform, "~> 1.0.0-rc8", override: true },
      { :conform_exrm, "~> 0.2.0" },
      { :dialyxir, "~> 0.2", only: ~w(dev test)a },
      { :dogma, "~> 0.0", only: ~w(dev test)a },
      { :exrm, "~> 1.0.0-rc7", override: true },
      { :insert_ordered_set, "~> 0.0.1" },
      { :kafka_ex, "0.3.0" },
      { :mongodb, "0.1.1" },
      { :poison, "~> 1.5" },
      { :poolboy, "~> 1.5.1" },
      { :timex, "~> 1.0.0-rc4" },
    ]
  end

[bitwalker]

Very strange, your process works for me locally (I have a little test app which I dumped your config in):

⟩ /tmp/test/current/bin/test console
Using /tmp/test/current/releases/0.0.3/test.sh
copying /tmp/test/current/releases/0.0.3/test.conf to /tmp/test/current/running-config/test.conf ...
using /tmp/test/current/running-config/test.conf to populate "/tmp/test/current/running-config".
Exec: /tmp/test/current/erts-7.2.1/bin/erlexec -boot /tmp/test/current/releases/0.0.3/test -boot_var ERTS_LIB_DIR /tmp/test/current/erts-7.2.1/../lib -env ERL_LIBS /tmp/test/current/lib -config /tmp/test/current/running-config/sys.config -args_file /tmp/test/current/running-config/vm.args -conform_schema /tmp/test/current/releases/0.0.3/test.schema.exs -conform_config /tmp/test/current/releases/0.0.3/test.conf -running_conf /tmp/test/current/running-config/test.conf -user Elixir.IEx.CLI -extra --no-halt +iex -- console
Root: /tmp/test/current
/tmp/test/current
Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Interactive Elixir (1.2.0) - press Ctrl+C to exit (type h() ENTER for help)
iex([email protected])1> Application.get_all_env(:test)
[env: :prod, another_val: "none",
 brokers: [{"192.168.99.101", "9092"}, {"192.168.99.104", "9092"},
  {"192.168.99.107", "9092"}], debug_level: :info, included_applications: []]
iex([email protected])2>
BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded
       (v)ersion (k)ill (D)b-tables (d)istribution
a

Only difference being that I changed kafka_ex to test in the schema and conf. What happens when you run the following?

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform

You should see the output below if it's working properly:

Conform - Translate the provided .conf file to a .config file using the given schema
-------
usage: conform --conf foo.conf --schema foo.schema.exs [options]

Options:
  --filename <name>:    Names the output file <name>.config
  --output-dir <path>:  Outputs the .config file to <path>/<sys|name>.config
  --config <config>:    Merges the translated configuration over the top of
                        <config> before output
  -h | --help:          Prints this help

What OS, Erlang and Elixir versions are you running on this machine?

[localshred]

Same result, undefined function:

$ /app/gretel/current/erts-7.2/bin/escript /app/gretel/current/releases/0.0.1/conform
escript: exception error: undefined function conform_escript:main/1
  in function  escript:run/2 (escript.erl, line 757)
  in call from escript:start/1 (escript.erl, line 277)
  in call from init:start_it/1
  in call from init:start_em/1

$ erl
Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]

$ elixir -v
Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false]

Elixir 1.2.0

$ uname -a
Linux gretel-stage-nuvis01 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux