Tracking specific environment variables about clearml

[bmartinn]

@eyalto How about this one (it will work in tandem with ArgParse, absl etc.):
(it is basically another set of parameters dictionary, we connect with the experiment/task)

import os
from trains import Task
task = Task.init(project_name='examples', task_name='os environment example')
task.connect({k: os.environ.get(k) for k in ['PATH', 'PWD', 'PYTHONPATH']})

[eyalto]

@bmartinn -

This indeed results in the same effect but I was looking for a mechanism that is external to the code that can be configured somehow externally and minimises the footprint in the code
.
This would be a way to track instrumentation of the external environment to the training job.
Things like dedicated links to additional tools (like job-id or link to the container running the training job - or other resources generated by the external environment)

I will take your advise and implement such mechanism inside our code - but it may be useful for other users / developers as well - if this is indeed the case ....

[bmartinn]

@eyalto thank you for the feature suggestion.
Here is what I was thinking:

$ export TRAINS_LOG_ENVIRONMENT=
(default) TRAINS does not log OS environment variables

$ export TRAINS_LOG_ENVIRONMENT="PWD,PYTHONPATH"
Will result in TRAINS logging values of both PWD and PYTHONPATH

$ export TRAINS_LOG_ENVIRONMENT="*"
Will result in TRAINS logging values of ALL environment variables

Is that what you had in mind?

[eyalto]

This is even better than what I was thinking about Will this be applied soon ?

…

On Mon, Jul 1, 2019, 22:44 Martin.B ***@***.***> wrote: @eyalto <https://github.com/eyalto> thank you for the feature suggestion. Here is what I was thinking: $ export TRAINS_LOG_ENVIRONMENT= (default) *TRAINS* does not log OS environment variables $ export TRAINS_LOG_ENVIRONMENT="PWD,PYTHONPATH" Will result in *TRAINS* logging values of both PWD and PYTHONPATH $ export TRAINS_LOG_ENVIRONMENT="*" Will result in *TRAINS* logging values of ALL environment variables Is that what you had in mind? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAGQ7EDGS7IOWMF7BZ3OOQLP5JNADANCNFSM4H4VLEQA> .

[bmartinn]

@eyalto, the feature is included in the latest release v0.10.2 exactly as suggested.

[shaulimaya]

Hey!
Thanks for the feature!
I already start using it, but I think that this is a little bit strange that the env params are under HYPER PARAMETERS
Maybe we can find other place for it?
Env params?

[bmartinn]

@shaulimaya great to hear you are already using this feature.
Yes, in the next version we will have a separate section for Environment Variables, until then they are under Hyper-Parameters.

[shaulimaya]

Thanks a lot!
Another small thing - if you can render links "http://blabla" as link
:)

[bmartinn]

@shaulimaya not sure security wise, this is recommended :)

[bmartinn]

Closing, feature is now available in 0.10.2