Kamal requires deploy.yml even when using explicit destination with -d about kamal

[ruyrocha]

It seems destination is missing in your case, and that would justify the exception. Please just hold on for a few while I'm checking into this issue.

[cosmin-marginean]

I've tried this below, because I need the -d to pick up the right .env file, but it fails

kamal env push -c config/deploy.test.yml -d test
...
ERROR (RuntimeError): Configuration file not found in [REDACTED]/config/deploy.test.test.yml

It seems to search for deploy.test.test.yml which is likely caused by -d test on top of -c.

Then this one fails as well, as the -c seems to override the -d

kamal env push -c config/deploy.yml -d test
...
ERROR (RuntimeError): Configuration file not found in [REDACTED]/config/deploy.yml

This one "works" in the sense that it doesn't fail with that error, but the correct .env is not being picked up (because of the lack of -d I'm assuming).

kamal env push -c config/deploy.test.yml

In any case, the behaviour of -d, according to this is that it picks up the right deploy.xxx.yml, which is what we'd expect. It also picks up the right .env file, which is documented separately here (See "Using a generated .env file").

This would make the need for a -c superfluous in this particular case, or one should take precedence over the other. Either way, it seems that the behaviour of -c and -d need to be aligned/reconsidered?

[nickhammond]

@cosmin-marginean You can have an empty deploy.<destination>.yml by just putting {} in that file. Or anything that Ruby can easily parse into a hash from YAML. This would let you do the last part that you're mentioning while still loading the specific .env.destination file.

[ruyrocha]

@cosmin-marginean it seems that you'll need to set the configuration file on command line if you're using different files on different environments, otherwise it's using the default value (./config/deploy.yml) and you'll always face that exception. Can you please give it a try?

[ruyrocha]

That error kinda makes sense.

It seems require_destination was introduced on #451, so can you please add it to your base config.yml like this? And then just pass the destination on command line.

[cosmin-marginean]

Ok, so in my base deploy.yml I have nothing but

require_destination: true

This seems to do the trick, so it's probably worth documenting this. I think there is still confusion around why a deploy.yml would be loaded at all, when we specify -d test. It would also be good to know whether settings in deploy.test.yml, for example, would override deploy.yml? In which case this would make sense, but I can't tell if that's the intention.

There's then the usecase when we only need one deploy.yml and different .env files (.env.test, .env.prod, etc). This would be useful, but it isn't possible because as soon as you specify -d test it needs a deploy.test.yml AND a deploy.yml :)

So I still feel like we're going around in circles, but at least I know how to work around it for now, and I appreciate your help!.

[nickhammond]

It would also be good to know whether settings in deploy.test.yml, for example, would override deploy.yml? In which case this would make sense, but I can't tell if that's the intention.

They do. Your destination specific file gets applied last. This is helpful so that you can supply common deploy options in deploy.yml(service, image, registry, etc.), and then destination-specific options within deploy.test.yml. This layering definitely comes from the Rails and Capistrano world since they behave pretty similarly with environments.

Do you have a completely different flow to deploy to test vs. production?

[cosmin-marginean]

@nickhammond Yes, that makes sense, thank you for confirming.

Do you have a completely different flow to deploy to test vs. production?

No, they're identical (just the .envs are different) and that's what I was trying to figure out, in order to minimise the number of duplicate content here.
From what I've seen a deploy.test.yml is required when using -d test as much as deploy.yml, even when we set require_destination: false. (And we need -d test to load .env.<destination> so that can't be avoided)

So the deploy.<destination>.yml files need to exist, and unfortunately, they can't be empty either (assuming the deploy.yml contains all the config we need). Leaving us with the only option to have:

1. A deploy.yml that only has require_destination: true (this can't be empty either)
2. A deploy.<destination>.yml for each of the destinations, which are identical/duplicate.

So I guess the use case is that we need different .env.<destination>, but we don't need different deploy.<destination>.yml. Having deploy.<destination>.yml present but empty (in case we want to override something in the future) is a reasonable compromise, but that's not possible.

[cosmin-marginean]

Ah, that is grand, I didn't know that! It works fine, thank you so much!

[nickhammond]

@cosmin-marginean Enjoy! Can this be closed?

[cosmin-marginean]

Yep, closing. Thank you all for your help