Fix the overwriting of nested parameters with `OmegaConfigLoader` about kedro

[noklam]

[astrojuanlu]

https://linen-slack.kedro.org/t/16184322/hi-team-a-question-when-using-kedro-params-at-runtime-right-#ad3c8635-2a81-45be-8242-349287387628

[noklam]

A more updated fork seems to suggest this is a bug. https://github.com/chy111126/debug_kedro_0182

The problem is the use of env and CLI --params is interacting weirdly.

1. kedro run --env=add --params="aaa.bbb.abb=2023-11-11"

{'aaa': {'bbb': {'abb': '2023-11-11'}}, 'xyz': {'asdf': 123123}, 'def': {'gg': 123}}

2. kedro run --params="aaa.bbb.abb=2023-11-11"

{'aaa': {'bbb': {'aba': '2023-11-01', 'abb': '2023-11-11', 'abc': 14}}, 'xyz': {'asdf': 123123}}

The add env only contains one top key def parameter which is not a conflicted key, but the result override the top level key aaa.

[noklam]

Further testing, I try to switch to ConfigLoader with 0.18.14 and the issue is gone. I did another test to upgrade to 0.19.1 with OmegaConfig and the issue is back.

[noklam]

A temporary workaround is to use settings.py, note this shouldn't be needed because the top-level key has no conflict. We should still investigate the issue.

CONFIG_LOADER_ARGS = {
    "merge_strategy": {"parameters": "destructive"},
}

[noklam]

config={'aaa': {'bbb': {'aba': '2023-11-01', 'abb': '2023-11-11', 'abc': 14}}, 'xyz': {'asdf': 123123}}
env_config={'def': {'gg': 123}, 'aaa': {'bbb': {'abb': '2023-11-11'}}}
resulting_config={'aaa': {'bbb': {'abb': '2023-11-11'}}, 'xyz': {'asdf': 123123}, 'def': {'gg': 123}}

It appears that the --params="aaa.bbb.abb=2023-11-11" somehow get pass into to the add environment as parameters, thus the destructive merge overriding it.