Goreviewpartner (GRP) , automated review tool, now supports phoenixgo native engine !! about phoenixgo

[pnprog]

Thanks,

I can confirm there are some issues with GRP there. For example, move 27 has two variations for o17, which obviously is wrong. The issue is most probably linked with GRP parsing all the log from PhoenixGo.

I will have to fix this before we start comparing both engines. There may be other issues to fix also :)

[pnprog]

Maybe some "edge cases" that appear only when a lot of playout are performed: Maybe the order in which the variations are displayed is not consistent, and GRP got confused or something like this.

I will try to have my computer analyse that move 27 during the whole night, and maybe I will find something inside.

[alreadydone]

@godmoves (of HappyGo expertise) might want to tune in. (See also pnprog/goreviewpartner#96)

[pnprog]

@wonderingabout

i'll rerun an analysis and we'll see if its better

I just pushed some changes on GRP repository. This should fix the issue we saw above, and also add information about number of playouts (in fact, number of visits). And also use less memory.

Could you have a try again now with that same game, so that we can compare the winrate curves? Don't hesitate to go with bigger tree size (maybe width=6, depth=10?) I am quite confident GRP should handle that properly now.

[wonderingabout]

@pnprog

ok will start it now then 👍

but lets just start with 6-6 for starters :)

also one thing i noticed when i close grp is that i'm not in contributors list :)
i dont mind much, but i think you would care more about it than me, am i right :)

[wonderingabout]

so here is the result 👍

http://www.mediafire.com/file/dvp18509dtj5eru/game2v2.rsgf/file

still some things to fix :)

[wonderingabout]

i think we can close this issue now,

support questions and research can continue in : pnprog/goreviewpartner#99

[wonderingabout]

example of moves analyzed by native PhoenixGo :

[wonderingabout]

compare it to LZ-phoenixgo-orig : https://github.com/alreadydone/lz/tree/phoenix-orig
grp release 0.14.2

[pnprog]

Hi @wonderingabout ,

Can you share both RSGF files for me to check? I am not sure why this "yoyo" effect. If it was a consequence of "100 - x", the beginning of the graph (left part) would be different for both engines, but this is not the case. So it must me some other issue.

A "yoyo" effect can be normal, when both players keep failing to see (and play) a game reversing move. But in this case, both engines should have the same result.

[wonderingabout]

@pnprog

here it is 👍

grp versions are different

grp cloned from github : http://www.mediafire.com/file/bubs1wi7x86zbk3/game2nativePhoenixGo.rsgf/file
grp release 14.2 : http://www.mediafire.com/file/dtw8taad9hg2qeo/game2LZPG.rsgf/file

also one bad behaviour i noticed is its possible for white to play in an already played black stone !!

[wonderingabout]

we can use this game as a benchmark when you release some fixes

i'll rerun an analysis and we'll see if its better

[wonderingabout]

take it easy :)

[wonderingabout]

@pnprog

gtx1060 power capped at 75W for low temps 👍

[wonderingabout]

also 2 : i'll increase sims to 3200 to have equal comparison with LZ-PhoenixGo

and also 3 : do i have to use --logtostderr in conf ?

[pnprog]

yes, logtostderr and verbose 1 are needed.

[wonderingabout]

i ask because i saw that you removed --logtostderr here :
(i used it everytime though)

https://github.com/pnprog/goreviewpartner/blob/master/config.ini#L100-L104

since phoenixgo can be a bit hard to configure, can i suggest that we add 2 examples here (1 for tensorrt, and 1 for notensorrt)

[PhoenixGo-0]
profile = Example-tensorrt
command = /home/yourusername/PhoenixGo/bazel-bin/mcts/mcts_main
parameters = --gtp --config_path=/home/yourusername/PhoenixGo/etc/mcts_1gpu_grp.conf --logtostderr --v=1
timepermove = 120

profile = Example-notensorrt-linux
command = /home/yourusername/PhoenixGo/bazel-bin/mcts/mcts_main
parameters = --gtp --config_path=/home/yourusername/PhoenixGo/etc/mcts_1gpu_notensorrt_grp.conf --logtostderr --v=1
timepermove = 120

profile = Example-notensorrt-windows
command = C:\users\yourusername\Downloads\PhoenixGo\bin\mcts_main
parameters = --gtp --config_path C:\users\yourusername\Downloads\PhoenixGo\bin\mcts_main --logtostderr --v 1
timepermove = 120

i'm sure these examples will be helpful for interested people, because there are a lot of things to pay attention to

i'll also add 2 new config files optimized for grp on phoenixgo github :

• remove time management to use fixed visits
• remove all other time management options
• mention that it is just an example
• add the debug @alreadydone mentionned, with example of 6x6

also, i would like if you sent a PR to my branch, after all you really helped phoenixgo be better

maybe i can commit with your username and email ? (i never tried it before so i wonder if it will work with fake username and email)

[wonderingabout]

btw time where i am now :

[pnprog]

i ask because i saw that you removed --logtostderr here

I haven't think too much about this. On my own set up, I set the command directly to the start.sh file, that's why (and maybe that's not the best idea).

For the moment, the entry is just called "Test" or something, because there are still things I need to test. Like handicap games (apparently PhoenixGo support up to 5 stones), and if the GTP time_setting command can really be used (in that case, I will add an entry in the setting panel just like Leela).

When everything is more or less ok, I will create two entries (slow and fast) like the other bots.

So for the next release of GRP (hopefully around February 10th) I will package one download for PhoenixGo (for Windows) already configured, and the weights file included.
And because everyone's hardware is different, there will be 3 downloads, one for GPU, one for CUDA's GPU, and one for CPU. So most of Windows users will be covered.
For other users, I will detail in details in the documentation what settings are necessary to have GRP work with PhoenixGo.

Do you have pushed your mcts_1gpu_grp.conf files somewhere already?

[wonderingabout]

oh wow i'm so careless lol

it just finished but
forgot to use the new config file, gotta redo the analysis again lol

[wonderingabout]

oh no actually i really used the right config file, my bad

[wonderingabout]

handicap is a bad idea, zero bots are too strong, max playable is 2 (or 3 for 10k or weaker players, with custom config file)

i dont know about start.sh, but i like to have my own settings :) (especially on windows where you need special syntax)

[pnprog]

still some things to fix :)

Yeah :)

I'm moving the discussion back to GRP project, to avoid polluting PhoenixGo project with GRP's issues :)

[wonderingabout]

discussion continues here :

pnprog/goreviewpartner#99

[pnprog]

also, i would like if you sent a PR to my branch, after all you really helped phoenixgo be better

Here it is: https://github.com/pnprog/PhoenixGo/tree/patch-3

In the context of GRP, because time is not critical (there is no risk of loosing a game on time) it is better to let the engine thinks more time if it can help it figure out better the best play. I do something similar for Pachi. In fact, I would go as far as increasing the time_factor up to 1.0 (0.3 is the default value).

I don't know how to PR to your branch, github only proposes me to PR to Tencent's master branch at this point.

By the way, I did not see a mcts_cpu_grp.conf file :)
And that's a lot of modifications to PhoenixGo's project, that are related to an other project, so they may not be interested in it anyway :)

[wonderingabout]

also, i would like if you sent a PR to my branch, after all you really helped phoenixgo be better

Here it is: https://github.com/pnprog/PhoenixGo/tree/patch-3

great 👍 @pnprog

now you need to go to pull requests of my fork here :

https://github.com/wonderingabout/PhoenixGo/pulls

then create new pull request

compare :

• wonderingabout/PhoenixGo - branch faqv2-bazel-master

with

• pnprog/PhoenixGo - branch patch-3

i will merge you pull request into my branch when you create the pull request 👍

[wonderingabout]

to answer your questions now, the grp files i created have time management all disabled (unlimited time), (all enable time options are set to 0), and timeouts set to 0 ms (infinite time)

infinite time :

timeout_ms_per_step: 0

unlimited time :

early_stop {
    enable: 0
    check_every_ms: 100
    sims_factor: 1.0
    sims_threshold: 2000
}
unstable_overtime {
    enable: 0
    time_factor: 0.3
}
behind_overtime {
    enable: 0
    act_threshold: 0.0
    time_factor: 0.3
}
time_control {
    enable: 0
    c_denom: 20
    c_maxply: 40
    reserved_time: 1.0
}

i forgot indeed the cpu version (and the cpu profile too, i thought it was included in notensorrt 😆 ), can you use this opportunity to send it to me ?

[pnprog]

I can't find wonderingabout/PhoenixGo - branch faqv2-bazel-master in the menu Choose a Base Repository, and I have no clue why. I am really helpless about all those github related things :(

[wonderingabout]

@pnprog

yes github is not intuitive to understand at firs, but we all went through that 👍

the problem you have is because you forked from Tencent/PhoenixGo, not from wonderingabout/PhoenixGo, so even though you managed to import my branch on your fork of Tencent/PhoenixGo, the automated pull requests point to Tencent/PhoenixGo where the faqv2-bazel branch doesnt exist

to fix that, i think you can write manually the repoowner/project e.g wonderingabout/PhoenixGo and find my branch in pull request

if not, then you can backup all your branches (download zip), then delete Tencent/PhoenixGo (in settings "dangerous zone" : https://help.github.com/articles/deleting-a-repository/ click to see screenshots)

then go to wonderingabout/PhoenixGo and fork it here "click fork" : https://github.com/wonderingabout/PhoenixGo