stjudecloud / oliver Goto Github PK
View Code? Open in Web Editor NEWAn opinionated Cromwell orchestration manager.
Home Page: https://stjudecloud.github.io/oliver
License: MIT License
An opinionated Cromwell orchestration manager.
Home Page: https://stjudecloud.github.io/oliver
License: MIT License
Did you have plans for these args Jobin?
oliver/oliver/subcommands/submit.py
Lines 67 to 68 in a161cf6
Describe the bug
when there is scatter loop in wdl script, error occurs.
To Reproduce
oliver su test_inspect.wdl
oliver i <wdl-id>
error output:
$ oliver i 38698c18-f3ca-492e-a7d7-d9a9bc1edcd7
[E 200422 01:34:28 errors:28] Expected key is missing! The code needs to be updated, please contact the author!
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7fbdb9855250>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7fbdb989ed70>, 2165426.275444354)]']
connector: <aiohttp.connector.TCPConnector object at 0x7fbdb9855590>
content of test_inspect.wdl
task t1 {
Boolean in
command {
echo ${in}
sleep 30
}
}
task sl {
Int s
command { sleep ${s} }
}
workflow run {
Boolean in=true
call t1 { input: in=in }
scatter (i in [1,2,3,4,5,6,7,8]) {
call sl { input: s = 120 }
}
}
oliver -h
description for oliver inputs
command says "outputs" when it should be "inputs"
Hey-o,
To help guide new people in a good direction, can cwl
and cwl-workflow
be removed from the keywords list for this repo as Cromwell has deprecated CWL for quite a while now?
Thanks!
Hello !
Describe the bug
Inline configuration parameters are not used when invoking some commands, such as batches
or status
It seems to affect the batch_interval_mins
parameter.
To Reproduce
Call:
oliver \
--cromwell-server https://aks-sciworkflow.sophiagenetics.com/cromwell/ \
--cromwell-api-version v1 \
--batch-interval-mins 2 \
batches
Output:
[E 220211 16:09:03 errors:28] The following required arguments are missing: batch_interval_mins!
We recommend you run 'oliver configure' to ensure all required arguments are cached in the oliver configuration file.
If you choose not to do this, you will need to pass their respective flags on the command line.
Expected behavior
Show job batches
Thanks for your help !
Describe the bug
oliver config set
on a fresh install without all 3(?) parameters fails because nothing is cached.
oliver i e52a9bdd-2d0c-452b-82d0-0073e0861805
Traceback (most recent call last):
File "/share/nastj10/B2C_RD_P2/USER/fuxiangke/software/Miniconda3/envs/cromwell/bin/oliver", line 10, in
sys.exit(main())
File "/share/nastj10/B2C_RD_P2/USER/fuxiangke/software/Miniconda3/envs/cromwell/lib/python3.7/site-packages/oliver/main.py", line 125, in main
asyncio.run(run())
File "/share/nastj10/B2C_RD_P2/USER/fuxiangke/software/Miniconda3/envs/cromwell/lib/python3.7/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/share/nastj10/B2C_RD_P2/USER/fuxiangke/software/Miniconda3/envs/cromwell/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
return future.result()
File "/share/nastj10/B2C_RD_P2/USER/fuxiangke/software/Miniconda3/envs/cromwell/lib/python3.7/site-packages/oliver/main.py", line 120, in run
await args["func"](args, cromwell)
File "/share/nastj10/B2C_RD_P2/USER/fuxiangke/software/Miniconda3/envs/cromwell/lib/python3.7/site-packages/oliver/subcommands/inspect.py", line 112, in call
calls = sorted(calls, key=lambda k: k["Start"])
TypeError: '<' not supported between instances of 'NoneType' and 'str'
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7f3101e84350>
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x7f3101e13130>, 53136171.49227757)]']
connector: <aiohttp.connector.TCPConnector object at 0x7f3101e84610>
can you give me some help? thank you very much!
i just configure as the cource or change the port, the same error:
Could not connect to http://localhost:5002. Is the Cromwell server reachable?
or Could not connect to http://localhost:8000. Is the Cromwell server reachable?
i don't know how to solve this problem,could you give me some advice,appreciate
Describe the bug
Dependency emits warning when running oliver aggregate
.
/home/afrantz/.miniconda3/envs/oliver/lib/python3.7/site-packages/pendulum/__init__.py:81: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html
obj = obj.zone
To Reproduce
Steps to reproduce the behavior:
oliver aggregate <foo>
Expected behavior
No warning
Is your feature request related to a problem? Please describe.
Seems oliver can not submit WDL jobs with dependencies. (multiple WDL files)
Describe the solution you'd like
oliver submit should have a new parameter, like --dependencies, so user can pointing it to the WDL zip file.
Describe alternatives you've considered
it would be better, if oliver could resolve the dependencies, then submit them all togeher automatically.
Hi ! First, thanks for the amazing work here ! The Python code is so smooth and clean. A real pleasure to read into that !
Our issue now ;) We deploy Cromwell in Kubernetes so when we want to access it, we are going through an Istio gateway like "https://our_host/cromwell/" (no port is specified). This way to access Cromwell is different than with a local instance where you have "localhost:PORT".
Hence, we do not want for the "urljoin" method in the api.py
to remove that "/cromwell" path. This should be linked to the fact that the Cromwell routes in the api.py
module are specified as absolute route (like "/api/workflows/...").
A possible fix could be to specify only relative route in the api.py
module and then to configure the "cromwell-api-server" with an url like "https://our_host/cromwell/" with a "/" at the end". Then it is working.
[oliver] docker run -it oliver:latest --cromwell-server "https://_our_host_/cromwell/" --cromwell-api-version "v1" status --batch-interval-mins "2" --debug
[D 211013 08:13:49 api:56] GET api/workflows/{version}/query
[D 211013 08:13:49 api:58] Access url: https://_our_host_/cromwell/api/workflows/v1/query
[I 211013 08:13:49 workflows:156] Found 7 eligible workflows given search criteria.
[D 211013 ...
An alternative could be to not use the "urljoin" method but to do something a bit more complex with "urlparse" or just to add a "/" between the bae path and the Cromwell route.
What do you think ? Did you ever encountered that use case before ?
Cheers !
Describe the bug
To Reproduce
Steps to reproduce the behavior:
$ pip install stjudecloud-oliver
$ oliver configure
Traceback (most recent call last):
File "/usr/local/bin/oliver", line 7, in <module>
from oliver.__main__ import main
File "/usr/local/lib/python3.6/site-packages/oliver/__main__.py", line 10, in <module>
from oliver.subcommands import (
File "/usr/local/lib/python3.6/site-packages/oliver/subcommands/abort.py", line 5, in <module>
from ..lib import api, reporting
File "/usr/local/lib/python3.6/site-packages/oliver/lib/reporting.py", line 3, in <module>
from typing import List, Dict, OrderedDict as OrderedDictType
ImportError: cannot import name 'OrderedDict'
Desktop (please complete the following information):
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.