Comments (5)
Did you get pass this error?
I am seeing a lot of message similar to yours, but I am uncertain what is causing this error or how to get passed it.
Tips are welcome!
JM
INFO:toil.leader:Issued job 'KtServerService' A/q/jobvU4WyV with job batch system ID: 3 and cores: 0, disk: 2.8 G, and memory: 50.3 G
INFO:toil.leader:Job ended successfully: 'KtServerService' A/q/jobvU4WyV
WARNING:toil.leader:The job seems to have left a log file, indicating failure: 'KtServerService' A/q/jobvU4WyV
WARNING:toil.leader:A/q/jobvU4WyV INFO:toil.worker:---TOIL WORKER OUTPUT LOG---
WARNING:toil.leader:A/q/jobvU4WyV INFO:toil:Running Toil version 3.19.0-0feb1d4d1b4fc66062fc4dbc5d8f7fb046df39e6.
WARNING:toil.leader:A/q/jobvU4WyV WARNING:toil.resource:'JTRES_487db7e35980364bd76c90413e988c94' may exist, but is not yet referenced by the worker (KeyError from os.environ[]).
WARNING:toil.leader:A/q/jobvU4WyV WARNING:toil.resource:'JTRES_487db7e35980364bd76c90413e988c94' may exist, but is not yet referenced by the worker (KeyError from os.environ[]).
WARNING:toil.leader:A/q/jobvU4WyV INFO:cactus.shared.common:Running the command ['netstat', '-tuplen']
WARNING:toil.leader:A/q/jobvU4WyV (No info could be read for "-p": geteuid()=56685 but you should be root.)
WARNING:toil.leader:A/q/jobvU4WyV INFO:cactus.shared.common:Running the command ['ktserver', '-port', '19362', '-ls', '-tout', '200000', '-th', '64', '-bgs', u'/tmp/toil-d26cb8ff-cabd-4073-9dff-c78622313dd7-24b9bf83-4de6-4cc2-806a-6f9b1f77386a/tmpsPKhar/e5afee6e-3501-486e-8a88-350bae38f4cf/tQlq7b6/snapshot', '-bgsc', 'lzo', '-bgsi', '1000000', '-log', u'/tmp/toil-d26cb8ff-cabd-4073-9dff-c78622313dd7-24b9bf83-4de6-4cc2-806a-6f9b1f77386a/tmpsPKhar/e5afee6e-3501-486e-8a88-350bae38f4cf/tmpXUtETZ.tmp', ':#opts=ls#bnum=30m#msiz=50g#ktopts=p']
WARNING:toil.leader:A/q/jobvU4WyV CRITICAL:toil.lib.bioio:Error starting ktserver.
WARNING:toil.leader:A/q/jobvU4WyV INFO:cactus.shared.common:Running the command ['ktremotemgr', 'remove', '-port', '19362', '-host', '10.31.138.118', 'TERMINATE']
WARNING:toil.leader:A/q/jobvU4WyV INFO:cactus.shared.common:Running the command ['ktremotemgr', 'get', '-port', '19362', '-host', '10.31.138.118', 'TERMINATE']
WARNING:toil.leader:A/q/jobvU4WyV Process ServerProcess-1:
WARNING:toil.leader:A/q/jobvU4WyV Traceback (most recent call last):
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
WARNING:toil.leader:A/q/jobvU4WyV self.run()
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/cactus/pipeline/ktserverControl.py", line 82, in run
WARNING:toil.leader:A/q/jobvU4WyV self.tryRun(*self.args, **self.kwargs)
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/cactus/pipeline/ktserverControl.py", line 118, in tryRun
WARNING:toil.leader:A/q/jobvU4WyV raise RuntimeError("KTServer failed. Log: %s" % f.read())
WARNING:toil.leader:A/q/jobvU4WyV RuntimeError: KTServer failed. Log: 2019-08-28T08:50:23.431847-05:00: [SYSTEM]: ================ [START]: pid=111118
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:23.432018-05:00: [SYSTEM]: opening a database: path=:#opts=ls#bnum=30m#msiz=50g#ktopts=p
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:23.432332-05:00: [SYSTEM]: applying a snapshot file: db=0 ts=1566409684634000000 count=20479544 size=15187944389
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618643-05:00: [ERROR]: [DB]: :: 9: system error: too short region
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618705-05:00: [ERROR]: could not apply a snapshot: system error: too short region
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618807-05:00: [SYSTEM]: starting the server: expr=:19362
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618869-05:00: [SYSTEM]: server socket opened: expr=:19362 timeout=200000.0
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618880-05:00: [SYSTEM]: listening server socket started: fd=4
WARNING:toil.leader:A/q/jobvU4WyV
WARNING:toil.leader:A/q/jobvU4WyV CRITICAL:toil.lib.bioio:Error starting ktserver.
WARNING:toil.leader:A/q/jobvU4WyV WARNING:toil.fileStore:LOG-TO-MASTER: Job used more disk than requested. Consider modifying the user script to avoid the chance of failure due to incorrectly requested resources. Job t/V/jobTdd5ip/g/tmpFoOJp5-_serialiseJob-stream used 401.26% (11.4 GB [12187840512B] used, 2.8 GB [3037418200B] requested) at the end of its run.
WARNING:toil.leader:A/q/jobvU4WyV Traceback (most recent call last):
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/toil/worker.py", line 324, in workerScript
WARNING:toil.leader:A/q/jobvU4WyV job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore)
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/toil/job.py", line 1351, in _runner
WARNING:toil.leader:A/q/jobvU4WyV returnValues = self._run(jobGraph, fileStore)
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/toil/job.py", line 1694, in _run
WARNING:toil.leader:A/q/jobvU4WyV returnValues = self.run(fileStore)
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/toil/job.py", line 1644, in run
WARNING:toil.leader:A/q/jobvU4WyV startCredentials = service.start(self)
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/cactus/pipeline/ktserverToil.py", line 33, in start
WARNING:toil.leader:A/q/jobvU4WyV snapshotExportID=snapshotExportID)
WARNING:toil.leader:A/q/jobvU4WyV File "/n/home01/lassance/.conda/envs/ENV_CACTUS/lib/python2.7/site-packages/cactus/pipeline/ktserverControl.py", line 62, in runKtserver
WARNING:toil.leader:A/q/jobvU4WyV raise RuntimeError("Unable to launch ktserver in time. Log: %s" % log)
WARNING:toil.leader:A/q/jobvU4WyV RuntimeError: Unable to launch ktserver in time. Log: 2019-08-28T08:50:23.431847-05:00: [SYSTEM]: ================ [START]: pid=111118
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:23.432018-05:00: [SYSTEM]: opening a database: path=:#opts=ls#bnum=30m#msiz=50g#ktopts=p
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:23.432332-05:00: [SYSTEM]: applying a snapshot file: db=0 ts=1566409684634000000 count=20479544 size=15187944389
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618643-05:00: [ERROR]: [DB]: :: 9: system error: too short region
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618705-05:00: [ERROR]: could not apply a snapshot: system error: too short region
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618807-05:00: [SYSTEM]: starting the server: expr=:19362
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618869-05:00: [SYSTEM]: server socket opened: expr=:19362 timeout=200000.0
WARNING:toil.leader:A/q/jobvU4WyV 2019-08-28T08:50:36.618880-05:00: [SYSTEM]: listening server socket started: fd=4
WARNING:toil.leader:A/q/jobvU4WyV
WARNING:toil.leader:A/q/jobvU4WyV ERROR:toil.worker:Exiting the worker because of a failed job on host holy2b17215.rc.fas.harvard.edu
WARNING:toil.leader:A/q/jobvU4WyV WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'KtServerService' A/q/jobvU4WyV with ID A/q/jobvU4WyV to 5
from cactus.
Hi @lassancejm, if I remember correctly, my solution was to run Cactus with singularity. It ran better this way. Sorry I can't help you more !
from cactus.
Hi @RomainFeron, thanks for posting about this. I'm receiving the same error as well. Unfortunately, we're running under Cactus under Singularity already... Do you happen to know what version of cactus you're using? (Looks like ours was installed on Oct 9, 2019...)
from cactus.
Hi @jasonsydes,
I have not tried to run Cactus since I posted this issue; I was cloning cactus from the git repository, so the version I ran was ~1 year older than yours.
from cactus.
Hi @RomainFeron, thanks for that info, that's helpful.
from cactus.
Related Issues (20)
- How compare mapping performance using vg-giraffe between pangenome and genomes used in pangenome construction HOT 1
- The sentence below, should`cactus-graphmap-join`be`cactus-pangenome`?
- Help parameterizing seqfile HOT 1
- Obtain node PAV, ID and fasta.
- unrecognized arguments: --bigChain HOT 1
- error cactus-align HOT 1
- unrecognized arguments: --maskAlpha HOT 4
- How to obtain the evolutionary order of root during merge hal file HOT 1
- Alt nodes
- ancestral reconstructed HOT 2
- Mapping settings
- HAL to vg for SV calling, vg index Assertion `idx < this->size()' failed HOT 1
- read input data (listed in seqfile) from local directory, not web location (url)? HOT 2
- Error in cactus-graphmap-join : ['vg clip -sS - -P CamPar;] exited 141: stderr=vg: invalid option -- 'S' HOT 1
- Segmental duplications HOT 1
- Minigraph-cactus VCF has overlapping variants after left align HOT 3
- Locations When Using cactus-hal2maf HOT 1
- CDS HOT 2
- cactus-graphmap-split taking a long time - is this expected? HOT 2
- toil issuing too many jobs at a time HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cactus.