Coder Social home page Coder Social logo

avail_mem not used for picard about chipseq HOT 5 CLOSED

nf-core avatar nf-core commented on May 29, 2024
avail_mem not used for picard

from chipseq.

Comments (5)

ewels avatar ewels commented on May 29, 2024 1

So basically, this code is no longer used for anything:

chipseq/main.nf

Lines 479 to 491 in 5f67d82

if( task.memory == null ){
log.warn "[Picard MarkDuplicates] Available memory not known - defaulting to 6GB ($prefix)"
avail_mem = 6000
} else {
avail_mem = task.memory.toMega()
if( avail_mem <= 0){
avail_mem = 6000
log.warn "[Picard MarkDuplicates] Available memory 0 - defaulting to 6GB ($prefix)"
} else if( avail_mem < 250){
avail_mem = 250
log.warn "[Picard MarkDuplicates] Available memory under 250MB - defaulting to 250MB ($prefix)"
}
}

You can see that previously we ran picard with that as the memory argument (java -Xmx${avail_mem}m), but that was dropped when we switched to the bioconda installation: 4bbf815#diff-28b7bea57b04f15beeb4c1de0ee78382L470

It's no big deal - just that we may as well delete the avail_mem snippet if it's not needed, or if we can specify the available memory then perhaps thats better? Not sure if it actually helps performance though. I think the bioconda script does a bunch of that anyway?

from chipseq.

apeltzer avatar apeltzer commented on May 29, 2024

Sorry, maybe I missed the discussion: Was there an issue using the one provided by picard in bioconda? Or what does this refer to?

from chipseq.

apeltzer avatar apeltzer commented on May 29, 2024

Okay, I missed that part. The bioconda recipe just adds a custom wrapper that makes it possible to run the Java Jar file with the specified amount of memory (e.g. see this script here which passes the command line parameters to the executable JAR file: https://github.com/bioconda/bioconda-recipes/blob/master/recipes/picard/picard.sh )

Not sure about the performance gain here either - also I found that "limiting Java applications" in terms of CPU/Memory isn't working as expectedly all the time. Some do quite a bunch of threading that looks to some schedulers as surpassing their limits unfortunately...

from chipseq.

ewels avatar ewels commented on May 29, 2024

x-ref nf-core/methylseq#45

Required code already present in the rnaseq pipeline, so should just be a copy & paste:

    script:
    if( !task.memory ){
        log.info "[Picard MarkDuplicates] Available memory not known - defaulting to 3GB. Specify process memory requirements to change this."
        avail_mem = 3
    } else {
        avail_mem = task.memory.toGiga()
    }
    """
    picard -Xmx${avail_mem}g MarkDuplicates \\

from chipseq.

drpatelh avatar drpatelh commented on May 29, 2024

Fixed in #61

from chipseq.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.