Error running long workflow about bioblend

[afgane]

Hi,
Does it always fail while trying to schedule the same job/tool? Can you
isolate that tool and see if a job with only it runs?

I don't have a good idea why this might be happening so we may need to
explore first a bit. requests library (used to perform the API calls)
returns the error too but that will require some changes in the code so we
may end up needing to try that too before it's resolved.
On Sep 11, 2013 6:00 PM, "hakattack" [email protected] wrote:

Hi,

I have ran long workflow (50 steps) and things seem to be working fine for
sometimes before it crashes. First all the data get loaded into the
library. The history gets created and then around 40/50 steps get scheduled
(looking at the new created history). The script keeps running but then it
crashes and here is the log:

Traceback (most recent call last):
File "run_imported_workflow.py", line 136, in
result = gi.workflows.run_workflow(workflow, datamap,
history_id=outputhist, import_inputs_to_history=True)
File "build/bdist.linux-x86_64/egg/bioblend/galaxy/workflows/init.py",
line 177, in run_workflow
File "build/bdist.linux-x86_64/egg/bioblend/galaxy/client.py", line 70, in
_post
File "build/bdist.linux-x86_64/egg/bioblend/galaxyclient.py", line 94, in
make_post_request
bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy:
500

My question, why only 40 not all the steps get scheduled? Is the cause for
the above issue relating to waiting time?

By the way, the workflow gets executed if I just ran from the Galaxy GUI.

Any help is highly appreciated.

—
Reply to this email directly or view it on GitHubhttps://github.com/afgane/bioblend/issues/41
.

[hakattack]

Thank you Enis for responding to my question.

Does it always fail while trying to schedule the same job/tool? Can you isolate that tool and see if a job with only it runs?

No it does not. Once I run the script “run_imported_workflow.py”, it does what is supposed to be doing. It loads the datasets, creates the history and then schedules the steps for the workflow. While still running I take a look at the created history and I only see the 40 steps out of 56 steps. The script keeps running for sometimes but then it dies out. It seems it keeps waiting for some response and then it crashes once it reach its waiting period.

I have been also trying the same workflow using the Galaxy “workflow_execute.py” API and had the same issue except that the script only schedule 15 steps rather than 40 steps (using bioblend).

I will be more than happy to share my workflow with you and then you can try it from your end and see if you get the same issue.

Best regards,

From: Enis Afgan [mailto:[email protected]]
Sent: Thursday, September 12, 2013 12:49 AM
To: afgane/bioblend
Cc: Hakeem Almabrazi
Subject: Re: [bioblend] Error running long workflow (#41)

Hi,
Does it always fail while trying to schedule the same job/tool? Can you
isolate that tool and see if a job with only it runs?

I don't have a good idea why this might be happening so we may need to
explore first a bit. requests library (used to perform the API calls)
returns the error too but that will require some changes in the code so we
may end up needing to try that too before it's resolved.
On Sep 11, 2013 6:00 PM, "hakattack" [email protected] wrote:

Hi,

I have ran long workflow (50 steps) and things seem to be working fine for
sometimes before it crashes. First all the data get loaded into the
library. The history gets created and then around 40/50 steps get scheduled
(looking at the new created history). The script keeps running but then it
crashes and here is the log:

Traceback (most recent call last):
File "run_imported_workflow.py", line 136, in
result = gi.workflows.run_workflow(workflow, datamap,
history_id=outputhist, import_inputs_to_history=True)
File "build/bdist.linux-x86_64/egg/bioblend/galaxy/workflows/init.py",
line 177, in run_workflow
File "build/bdist.linux-x86_64/egg/bioblend/galaxy/client.py", line 70, in
_post
File "build/bdist.linux-x86_64/egg/bioblend/galaxyclient.py", line 94, in
make_post_request
bioblend.galaxy.client.ConnectionError: Unexpected response from galaxy:
500

My question, why only 40 not all the steps get scheduled? Is the cause for
the above issue relating to waiting time?

By the way, the workflow gets executed if I just ran from the Galaxy GUI.

Any help is highly appreciated.

—
Reply to this email directly or view it on GitHubhttps://github.com/afgane/bioblend/issues/41
.

—
Reply to this email directly or view it on GitHubhttps://github.com/afgane/bioblend/issues/41#issuecomment-24296509.

[afgane]

I'm guessing you're using this workflow on your own instance of Galaxy? What is Galaxy's log showing when the script fails?
Depending on which tools/data you're using, sharing the workflow as an exported entity may be rather complicated to get running on an arbitrary Galaxy instance so let's see if we can resolve it on your instance for the time being.

[afgane]

Also, pull in the code from the tip of the BioBlend repo and run it. I just added the code to display the body of the error message in addition to the error code so maybe that will shed some light on the issue.

[hakattack]

Thank you Enis for your respond. I will give it a shot. I am very interested in making this work but I am a need to finish a project in hand. I will let you know what I find.

Regards,

Hakeem

From: Enis Afgan [mailto:[email protected]]
Sent: Monday, September 16, 2013 6:08 AM
To: afgane/bioblend
Cc: Hakeem Almabrazi
Subject: Re: [bioblend] Error running long workflow (#41)

Also, pull in the code from the tip of the BioBlend repo and run it. I just added the code to display the body of the error message in addition to the error code so maybe that will shed some light on the issue.

—
Reply to this email directly or view it on GitHubhttps://github.com/afgane/bioblend/issues/41#issuecomment-24502633.

[hakattack]

Hi Enis,

How are you doing? I just came back to this. I am trying to automate downloading files from Galaxy into local file system using your Bioblend API gi.datasets.download_dataset(dataset_id, file_path=None, use_default_filename=True, wait_for_completion=False, maxwait=12000)
From http://bioblend.readthedocs.org/en/latest/api_docs/galaxy/all.html#module-bioblend.galaxy.datasets but it does not seem to work for me. It returns empty string ‘’ when I ran it from the command line. The dataset I am using exists and available using the show_dataset.

Am I missing something here?

Thank you your time and help.

Hakeem

From: Enis Afgan [mailto:[email protected]]
Sent: Monday, September 16, 2013 6:08 AM
To: afgane/bioblend
Cc: Hakeem Almabrazi
Subject: Re: [bioblend] Error running long workflow (#41)

Also, pull in the code from the tip of the BioBlend repo and run it. I just added the code to display the body of the error message in addition to the error code so maybe that will shed some light on the issue.

—
Reply to this email directly or view it on GitHubhttps://github.com/afgane/bioblend/issues/41#issuecomment-24502633.

[nsoranzo]

Hi @hakattack,
download_dataset() method has been reworked in the mean time, is this still a problem for you?

If yes, can you please report it as another issue? This has been originally open for a different problem which seems to be fixed for you, so I'm closing this issue.