FSL melodic: review the launch form, and add documentation (explanations) wherever appropriate. about cbrain-plugins-neuro

[Najmahan]

1. the option to have three columns in the CSV for an intermediary registration to high-res fMRI is not supported
2. Launching FSLmelodic on Guillimin with FSL5 produced 132 "completed" fsl_sub tasks. ?

[glatard]

1. I don't think we have tracked this feature properly. Could you create an issue for it and provide test data? Is it required for the first public release or can it wait for the "big data" release?
2. Actually 1 completed FSL melodic task and 131 completed fsl_sub tasks. Is there any problem with that?

[Najmahan]

1. The issue was in the first user requirement document, I will create a
   new issue for it though. It is an issue encountered by people with small
   data, so I suggest we address it--even if as a limitation.
2. I think you are looking at a different project. If I launch the task
   from a project on which you are not a member, it fails on setting up
   prerequisites with Error "server Guillimin cannot synchronize content from
   provider ADNI-Results. I am triggering a "recovery" (for the second time,
   just in case this is pertinent to a glitch in the server yesterday.

On Thu, Jan 28, 2016 at 7:33 PM, Tristan Glatard [email protected]
wrote:

I don't think we have tracked this feature properly. Could you create
an issue for it and provide test data? Is it required for the first public
release or can it wait for the "big data" release?
2.

Actually 1 completed FSL melodic task and 131 completed fsl_sub tasks.
Is there any problem with that?

—
Reply to this email directly or view it on GitHub
#4 (comment)
.

[glatard]

1. OK, but it may introduce further delays before the public release (development, testing, deployment on the production server). I feel that we should aim at opening the tool asap so that we can collect feedback from external users.
2. I can look at all your tasks now (I have an admin account).
   a. On Jan 22nd, 132 of your tasks, including 1 melodic and 131 fsl_sub completed, and it doesn't look like there is any issue with these tasks.
   b. I'm not sure why these synchronisation issues happened. I'm going to investigate.
   c. Anyway, Melodic tasks are not restartable now (see #17). The fact that you were able to relaunch these tasks is a bug in the portal (see aces/cbrain#96). Once restarted, tasks will fail saying that they can't be restarted). You should create a new task. Note that you can save/load presets to avoid having to set all the parameters again.
   I think that such operational/support discussions should happen on the CBRAIN forum rather than on Github as they may be useful to other users who may experience similar issues. I'm going to follow-up on this issue there.

[Najmahan]

1. Public release will just force us to answer to many more of these
   questions that I am raising. But feel free.

a. You are not looking at correct tasks. Please see batch 309938, launched
on Jan 28.
b. it failed again
c. If such a bug exists, we should definitely NOT release this to the
public.

1. I am loosing track of where to discuss what.

On Fri, Jan 29, 2016 at 10:05 AM, Tristan Glatard [email protected]
wrote:

OK, but it may introduce further delays before the public release
(development, testing, deployment on the production server). I feel that we
should aim at opening the tool asap so that we can collect feedback from
external users.
2.

I can look at all your tasks now (I have an admin account).
a. On Jan 22nd, 132 of your tasks, including 1 melodic and 131 fsl_sub
completed, and it doesn't look like there is any issue with these tasks.
b. I'm not sure why these synchronisation issues happened. I'm going
to investigate.
c. Anyway, Melodic tasks are not restartable now (see #17
#17). The fact
that you were able to relaunch these tasks is a bug in the portal (see
aces/cbrain#96 aces/cbrain#96). Once
restarted, tasks will fail saying that they can't be restarted). You should
create a new task. Note that you can save/load presets to avoid having to
set all the parameters again.
I think that such operational/support discussions should happen on the
CBRAIN forum rather than on Github as they may be useful to other users who
may experience similar issues. I'm going to follow-up on this issue there.

—
Reply to this email directly or view it on GitHub
#4 (comment)
.

[glatard]

1. Fine, #21 is now added to the public release.
2. a. ok, I think we agree where the problem is :)
   b. please resubmit a new task (!= restarting the existing one).
   c. then, either aces/cbrain#96 should be put "High priority", or #17 should be moved to the public melodic release.
3. Github is for bug reports and feature requests. Forum is for user support.

[glatard]

(2.c. I just put #17 in the public melodic release)

[prioux]

@Najmahan The error message "server Guillimin cannot synchronize content from provider ADNI-Results" is supposed to appear when a server (Guillimin) is known not be be able to access a data provider (e.g. because of a firewall). In your case, the DP "ADNI-Results" is on one of your machines at the MNI and we know the firewall allows Guillimin to connect to it, and in CBRAIN it is recorded properly that Guillimin should be able to access it. So I don't get how you got that error message, because it would imply that CBRAIN thinks it cannot access it (without even trying, I might add). The only explanation I can think of is that yesterday it was set to 'cannot connect' and someone reset it since then.

[prioux]

If it happens again, please let me know right away.

[glatard]

@prioux, I just added Guillimin to the list of execution servers that can read from @Najmahan's DP. See post at https://forum.cbrain.mcgill.ca/t/error-server-xxx-cannot-synchronize-content-from-provider-yyy/38

[Najmahan]

Yes, that was my first thought and therefore I restarted it again today (I
learned that I should not have restarted--because it failed again, but
should have launched a new task).

On Fri, Jan 29, 2016 at 12:05 PM, Pierre Rioux [email protected]
wrote:

@Najmahan https://github.com/Najmahan The error message "server
Guillimin cannot synchronize content from provider ADNI-Results" is
supposed to appear when a server (Guillimin) is known not be be able to
access a data provider (e.g. because of a firewall). In your case, the DP
"ADNI-Results" is on one of your machines at the MNI and we know the
firewall allows Guillimin to connect to it, and in CBRAIN it is recorded
properly that Guillimin should be able to access it. So I don't get how you
got that error message, because it would imply that CBRAIN thinks it cannot
access it (without even trying, I might add). The only explanation I can
think of is that yesterday it was set to 'cannot connect' and someone reset
it since then.

—
Reply to this email directly or view it on GitHub
#4 (comment)
.

[prioux]

@glatard Yes, I just figured out your adjusted the checkbox, which was by mistake set to "false" since December. I had to look at the CBRAIN logs, and saw you did the update as admin this morning at 10:12. Ok thanks, mystery solved.

[Najmahan]

The workaround for doing Group-ICA on very large datasets is described in this paper
http://www.sciencedirect.com/science/article/pii/S105381191400634X

In Jan 25 2015, Steve Smith stated that this option is not the default yet (in the old version) and to invoke it, he suggested the following:

"probably the best solution is to use a new option for the group-PCA step in melodic.

This is called MIGP - it is virtually as accurate as doing all-subjects-temporally-concatenated (following by a huge PCA), but requires MUCH lower RAM requirements (that don't scale up with increasing subject numbers) and is not too slow (though with 140 subjects it could take a few days).

See paper at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4289914/

For default usage, change "-a concat" to:
--migp --sep_vn
and turn on -v so you can watch the progress.
"
This is not parallelized version, but at the end of the article a code for paralellization in matlab is provided.