The devDependency mocha was updated from 6.0.2 to 6.1.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

mocha is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

There have been updates to the babel7 monorepo:

• • The devDependency @babel/cli was updated from 7.7.7 to 7.8.0.

• The devDependency @babel/core was updated from 7.7.7 to 7.8.0.
• The devDependency @babel/plugin-transform-runtime was updated from 7.7.6 to 7.8.0.
• The devDependency @babel/preset-env was updated from 7.7.7 to 7.8.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

This monorepo update includes releases of one or more dependencies which all belong to the babel7 group definition.

babel7 is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

The devDependency eslint was updated from 5.15.3 to 5.16.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

eslint is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

This CRAM file with duplicates removed from @sashalau produced this sequence

The numbers in this list appear to be char codes from the ASCII table that correspond with ACGT

@jrobinso Reported that a CRAM file with very few reads could take longer than expected because it ended up fetching the entire chromosome

There are some areas with use of viewAsPairs where some excessive CPU usage is generated. The algorithm for determining which redispatch requests are made is sort of a brute force tool, calling getEntriesForRange for each unmatched read, and then it de-duplicates the results

In one area of a long insert size test file, I find this region of code comes up with https://github.com/GMOD/cram-js/blob/master/src/indexedCramFile.js#L94-L113

-541996 chunks before duplication
-32 after deduplication

This issue also applies to bam-js code

Should be able to pass an AbortSignal object to many of these calls to get them to abort.

In the CRAM spec this is identified as SUBEXP 7 int offset, int K subexponential coding

The 1000 genomes cram files on their servers use this codec

This data directory has some files that access the 1000 genomes server. Note that it doesn't contain a refseq track so I am not sure if that is a factor

1000genomes.tar.gz

Thoughts on supporting the browser File object? If you're not working on it, I might take a crack at it. It should be pretty simple, I think we need to implement an "io" file class for it, then some conditional logic to prevent the app from blowing up if run in node.

I'm trying to do some benchmarking of CRAM-JS and am having some trouble with the example code.

I have three cram files and their corresponding crai indices as well as the reference sequences in separate fasta files. I'd like to use the local reference sequences to read the cram files, but I'm not sure how to change the seqFetch argument to do this. Currently the code just seems to generate a fake sequence of A's and the documentation is not entirely clear on how to use the local sequence file.

Also there might be some bugs with fetch records portion of the code (e.g the await is not inside an async)

const { IndexedCramFile, CramFile, CraiIndex } = require('@gmod/cram')

// open local files
const indexedFile = new IndexedCramFile({
  cramPath: require.resolve(`./data/ce#5.tmp.cram`),
  index: new CraiIndex({
    path: require.resolve(`./data/ce#5.tmp.cram.crai`),
  }),
  seqFetch: async (seqId, start, end) => {
    // seqFetch should return a promise for a string.
    // this one just returns a fake sequence of all A's of the proper length
    let fakeSeq = ''
    for (let i = start; i <= end; i += 1) {
      fakeSeq += 'A'
    }
    return fakeSeq
  },
  checkSequenceMD5: false,
})

// example of fetching records from an indexed CRAM file.
// NOTE: only numeric IDs for the reference sequence are accepted
const records = await indexedFile.getRecordsForRange(0, 10000, 20000)
records.forEach(record => {
  console.log(`got a record named ${record.readName}`)
  record.readFeatures.forEach(({ code, pos, refPos, ref, sub }) => {
    if (code === 'X')
      console.log(
        `${
          record.readName
        } shows a base substitution of ${ref}->${sub} at ${refPos}`,
      )
  })
})


// can also pass `cramUrl`, and `url` params to open remote URLs

I've tried to export your CRAM and VCF modules with these tools with no luck. I've been able to browserify another VCF module, but I need your CRAM module to be used locally in the browser.

Have you tried this and do you know if this is possible? Thanks.

The devDependency prettier was updated from 1.17.1 to 1.18.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

prettier is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

The current commit for 1.5.6 and the github tag were not pushed during last release

We probably should fix our publish because this happens across all our packages

The module is now compiled with tsc, so incrementally, we can add typescript throughout the codebase for better dev experience for downstream consumers and to catch bugs

Recently I have opened an issue on IGV repository: igvteam/igv.js#1805 about a problem loading cram files. Initially, the problem was on the IGV tool side because there was a harded-coded limit related to the fetch size limit. They removed the limit, but now the cram file does not load. IGV developers recommended openning an issue here to report the problem. It seems that cram-js cannot load deep coverage cram files. I can send you the files I am trying to load, if necessary.

The devDependency webpack was updated from 4.35.3 to 4.36.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

webpack is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴

Dear @cmdcolin

We found another bug, if cram´s have been compressed additionally with the lzma. As the decompression is not available.

Maybe this is similar to the bzip2 issue, and the library just needs to be added? (https://github.com/LZMA-JS/LZMA-JS)

Found this file https://s3.amazonaws.com/jbrowse.org/genomes/hg19/hg002/HG002_ONTrel2_16x_RG_HP10xtrioRTG.cram.crai

It is actually a BAI file....even though it has crai extension

I have read that BAI can actually serve as a CRAI index, so we could consider supporting this, or perhaps consider throwing an error in this case?

Source:
ftp://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/AshkenazimTrio/HG002_NA24385_son/Ultralong_OxfordNanopore/combined_2018-08-10/

Apparently this is probably a GATK-ism https://sourceforge.net/p/samtools/mailman/message/34369364/

If I run yarn run tests I get 3 tests failed. Is that normal?

Hey guys, I found a bug that occurs with some webservers when running in a browser. I'm reporting the issue, and wondering if you've seen it in JBrowse? I have some URLs below that reproduce the problem.

The code below is lines 33 and 34 of remoteFile.js. sizeMatch can be null when 'content-range' is not returned in the headers, causing the next line to fail.

  // try to parse out the size of the remote file
  const sizeMatch = /\/(\d+)$/.exec(response.headers.get('content-range'))
  if (sizeMatch[1]) this._stat = { size: parseInt(sizeMatch[1], 10) }

The test file I'm using is

const cramUrl =
  'https://data.broadinstitute.org/igvdata/test/data/cram/NA12878.cram'
const indexUrl =
  'https://data.broadinstitute.org/igvdata/test/data/cram/NA12878.crai'

Note that this error does not occur when running from node, only when running from a browser with a webpack converted bundle. Looking at the network traffic the 'content-range' header is returned by the server.

This is a follow up from the previous issue of #34

Currently when large blocks of sequence are loaded from 'b', we don't have SNPs in this file. I don't know if we need to manually compare it with a seqFetch to see if SNPs exist?

Working

samtools view https://s3.amazonaws.com/jbrowse.org/genomes/hg19/hg002/HG002_ONTrel2_16x_RG_HP10xtrioRTG.cram | less

In jbrowse 1

Error: invalid .crai index file

In jbrowse 2

Error: CRAM version 0 not supported

Also I console.logged the version it thinks it is getting and it's all corrupted

{magic: "����", majorVersion: 0, minorVersion: 0, fileId: "��BC�t��Z]���U~"}
magic: "����"
majorVersion: 0
minorVersion: 0
fileId: "��BC�t��Z]���U~"

Not sure why the variability

https://github.com/jkbonfield/htscodecs/tree/master/javascript

I was wondering if it was possible to estimate template length for reads in more cases.

The current code seems to estimate smaller template lengths well but seems to struggle a bit on larger size ones

Example cram data attached with a test.js script that outputs the templateLength. Many are undefined except for shorter ones

large_inserts_data.tar.gz

Samtools view on the cram file with cut -f9 will be the template length too for comparison

With the switch to quick-lru, the feature cache can now become too big. This needs fixing.

introduced by cf56ed2#diff-ac236967b94260ff62d11f891c75549dL46

There have been updates to the babel7 monorepo:

• • The devDependency @babel/cli was updated from 7.8.3 to 7.8.4.

• The devDependency @babel/core was updated from 7.8.4 to 7.8.6.
• The devDependency @babel/plugin-transform-runtime was updated from 7.8.0 to 7.8.3.
• The devDependency @babel/preset-env was updated from 7.8.4 to 7.8.6.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

This monorepo update includes releases of one or more dependencies which all belong to the babel7 group definition.

babel7 is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Your Greenkeeper Bot 🌴