Comments (17)
I've copied the two VCFs to /lustre/scratch118/malaria/team112/personal/gg18/phasing-validation - let you get started while we figure out the access issues with gs://malariagen/
from pipelines.
Hi @gbggrant , just wondering who owns the gs://malariagen bucket? I can see files, but I can't download them.
Copying gs://malariagen/genetic_maps/ag1000g/phase2/AR1/X.gmap...
AccessDeniedException: 403 HttpError accessing <https://storage.googleapis.com/download/storage/v1/b/malariagen/o/genetic_maps%2Fag1000g%2Fphase2%2FAR1%2F2R.gmap?generation=1607974239579471&alt=media>: response: <{'x-guploader-uploadid': 'ABg5-Uy2UzVpHMg0YylQMsF7pOCnc8EsXngQ6KQ2lfKA06AVXnb0CBXASi0b0kznLyBwe2wB6U9DuHSLvCuWWDb1zko', 'content-type': 'text/html; charset=UTF-8', 'date': 'Mon, 11 Jan 2021 14:17:04 GMT', 'vary': 'Origin, X-Origin', 'expires': 'Mon, 11 Jan 2021 14:17:04 GMT', 'cache-control': 'private, max-age=0', 'content-length': '106', 'server': 'UploadServer', 'status': '403'}>, content <[email protected] does not have storage.objects.get access to the Google Cloud Storage object.>
from pipelines.
@hardingnj access is managed by a google group. 'malariagen' which you are in (under [email protected]). Can you confirm that's the user you are trying to access the files as?
from pipelines.
I'm pretty sure (unless I am misunderstanding something). That's the email in the error message above at least
If I switch to my personal account, I can't even ls:
(base) njh@debian:~$ gcloud auth list
Credentialed Accounts
ACTIVE ACCOUNT
[email protected]
* [email protected]
To set the active account, run:
$ gcloud config set account `ACCOUNT`
(base) njh@debian:~$ gsutil ls gs://malariagen
AccessDeniedException: 403 [email protected] does not have storage.objects.list access to the Google Cloud Storage bucket.
from pipelines.
okay - it must be something about the way we have it configured.
from pipelines.
@hardingnj can you give it another try - we've just updated permissions here. No worries if you don't get to this until tomorrow.
from pipelines.
Still a 403 unfortunately...
from pipelines.
Dang. Using the [email protected] I assume? (the gmail hasn't been added to the malariagen group).
from pipelines.
Dang. Using the [email protected] I assume? (the gmail hasn't been added to the malariagen group).
Unfortunately so!
from pipelines.
Hi again @hardingnj - we've updated read permissions on that bucket (gs://malariagen) again. Can you try to download data again and see if it works?
from pipelines.
That's done it. Thanks!
from pipelines.
Great! Sorry for the problems.
from pipelines.
Results look good. A more detailed PR is referenced below in vector-ops
Chromosome plots of the h12 summary statistic.
The phase 2 plots of the GSTE locus (top) compared to the new pipeline (bottom)
from pipelines.
Hi all, just to confirm that the results from @hardingnj are excellent. Previous analyses replicate extremely well with the haplotypes from the new pipeline. I think this gives us all the confidence we need to go ahead with the new pipeline.
from pipelines.
Great- excited to see this going so well- looking forward to analysing the whole cohort!
from pipelines.
Reopening this issue to record results from a second round of validation against the pipeline with genome region scatter and ligation.
From @jessicaway on 1 April:
The phasing outputs for the 167 sample validation set are available at gs://malariagen/Phasing/validation/Ag1000Phase2_BurkinaFaso/ (these were run with the genome region scatter and ligation).
I've rerun the validation analysis against these new results and everything looks great. Here's the results of H12 selection scans, comparing the new pipeline ("dev-release-2") against the Ag1000G phase 2 haplotypes:
Happy to sign off on the new pipeline implementation, sorry it took so long!
from pipelines.
xref https://github.com/malariagen/vector-ops/pull/1402
from pipelines.
Related Issues (20)
- Number of SNPs and Indels doesn't match with the numbers reported in README files HOT 4
- Provide example crams released by Core Sequencing HOT 3
- Define read group contents in vector short read alignment pipelines HOT 1
- Input source for mosquito short read alignment pipeline HOT 3
- Validation/truth datasets for variant calling calibration, comparison and evaluation HOT 26
- Copy script to convert vcf to zarr to this repo
- Port baseline Plasmodium amplicon SNP calling pipeline. HOT 1
- Add platform argument to build_pipeline_release script HOT 2
- How to make bwa short read alignment deterministic for testing HOT 1
- Mosquito phasing pipeline development HOT 22
- Genotype Concordance Validation of genotyping pipelines HOT 10
- Create pipelines that download multiple samples from IRODS in a batch
- Determine final lanelet manifest fields required for vector alignment and genotyping HOT 1
- Cromwell configurations on farm5 are using localization = copy, which explodes size of workflow working directory
- Convert cram to bam in vector alignment pipeline
- malaria-dk lsf group deprecated
- Output of vector short read alignment pipeline is named *.bam.bam HOT 1
- Use flexible fields in manifest
- Ag3.0 phasing pipeline production run HOT 15
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pipelines.