Running fc-solve's py-trains-test.py does not yield any result after a > 6 hours wait about clearml

[shlomif]

On Sat, 20 Jul 2019 14:30:26 -0700 "Martin.B" ***@***.***> wrote: @shlomif I think there is a bit of a misunderstanding on what _TRAINS_ was built for. _TRAINS_ is all about experiment versioning and auditing, and it achieves its "almost zero" integration by connecting to a predefined list of frameworks (currently TensorFlow/Keras/PyTorch/Matplotlib/SeaBorn etc.). When used with other/custom code, what you can expect from out-of-the-box usage is: Log of git commits (and git diff) Log of used python packages All the rest, i.e. graphs, model auditing etc. relies on the use of these frameworks, since in your case it is all custom code, I can't see how _TRAINS_ can magically add graphs and model auditing :) That said, I would suggest you manually add a few graphs to your code, using matplotlib, and see how _TRAINS_ automatically adds them to your experiment logging. I'm hoping it will be helpful in your algorithm optimization efforts.

I see - thanks for the clarification. I'll see what I can do.

…

-- ----------------------------------------------------------------- Shlomi Fish http://www.shlomifish.org/ What does “Zionism” mean? - http://shlom.in/def-zionism The cool thing about Vim is — you find something interesting with every typo. — Su‐Shee on Freenode’s #perl . Please reply to list if it's a mailing list post - http://shlom.in/reply .

[shlomif]

OK, I simplified getting the program to run on the git master branch. Now I will work on a docker program.

[shlomif]

See https://github.com/shlomif/fc-solve/blob/master/fc-solve/scripts/docker--py-trains.pl for getting the program to run.

[bmartinn]

I looked at the code, what exactly are you expecting to see while running it (other than of course the std output)?
I could not detect a Tensorboard or matplotlib graph being used, but I might have missed it.

[shlomif]

I looked at the code, what exactly are you expecting to see while running it (other than of course the std output)?

I generates initial freecell ms deals and solves them using a fast-on-average search and emits the moves of the solution. See https://fc-solve.shlomifish.org/faq.html for more info. I hope to get some insights about an algo to solve freecell, or at least some recommendations for improvement.

I could not detect a Tensorboard or matplotlib graph being used, but I might have missed it.

I did not use them. Should I?

[bmartinn]

You mentioned:

a record for it is created on the test server, but without any result after a more than 6 hours wait.

How is this record (I assume a file) created? Are you using Tensorflow/Keras/PyTorch?
These are currently the supported frameworks, where Trains auto-magically detects/registers/upload a model file.

[shlomif]

Hi,

You mentioned:

a record for it is created on the test server, but without any result after a more than 6 hours wait.

How is this record (I assume a file) created?

It is a link to a page of the task on the server given in the output on the terminal. See:

shlomif[fcs]:$trunk/fc-solve/source$ PYTHONPATH=$PWD/t/lib python3 ../scripts/py-trains-test.py | ( head -20 ; cat > /dev/null)
TRAINS Task: overwriting (reusing) task id=eed20e73cc4749c387e595c2c336ca9c
TRAINS-SERVER new version available: upgrade to v0.10.2 is recommended!
TRAINS results page: https://demoapp.trainsai.io/projects/2d3bc0da63204693bea85d346ac91b9f/experiments/eed20e73cc4749c387e595c2c336ca9c/output/log
JD KD 2S 4C 3S 6D 6S
2D KC KS 5C TD 8S 9C
9H 9S 9D TS 4S 8D 2H
JC 5S QD QH TH QS 6H
5D AD JS 4H 8H 6C
7H QC AS AC 2C 3D
7C KH AH 4D JH 8C
5H 3H 3C 7S 7D TC
[1, 4, 0, 0]
[1, 4, 1, 0]
[1, 4, 2, 0]
[1, 4, 3, 0]
[4, 4, 2, 0]
[2, 1, 1, 0]
[1, 7, 1, 0]
[2, 0, 7, 0]
[1, 6, 0, 0]

I'm referring to https://demoapp.trainsai.io/projects/2d3bc0da63204693bea85d346ac91b9f/experiments/eed20e73cc4749c387e595c2c336ca9c/output/log .

Are you using Tensorflow/Keras/PyTorch?

I think I am not using them - at least not directly. They are not mentioned in the synopsis in the links from Shay's original post here: https://groups.google.com/forum/#!topic/pyweb-il/0UmR-YjabIo .

These are currently the supported frameworks, where Trains auto-magically detects/registers/upload a model file.

[bmartinn]

@shlomif I think there is a bit of a misunderstanding on what TRAINS was built for.
TRAINS is all about experiment versioning and auditing, and it achieves its "almost zero" integration by connecting to a predefined list of frameworks (currently TensorFlow/Keras/PyTorch/Matplotlib/SeaBorn etc.).

When used with other/custom code, what you can expect from out-of-the-box usage is:
Log of git commits (and git diff)
Log of used python packages

All the rest, i.e. graphs, model auditing etc. relies on the use of these frameworks, since in your case it is all custom code, I can't see how TRAINS can magically add graphs and model auditing :)

That said, I would suggest you manually add a few graphs to your code, using matplotlib, and see how TRAINS automatically adds them to your experiment logging.

I'm hoping it will be helpful in your algorithm optimization efforts.