percolator / percolator

Semi-supervised learning for peptide identification from shotgun proteomics datasets

Percolator

This repository holds the source code for percolator and format converters software packages; a software for postprocessing of shotgun proteomics data.

The source code is hosted at github, binary downloads can be found in the release repository: https://github.com/percolator/percolator/releases

The building procedure is cmake-based. Helper scripts for building on different platforms can be found in: admin/builders/

The cleanest way to build is to use the vagrant virtual machine system, which can be setup using the script: admin/vagrant/manager.sh

percolator's Issues

cmake: build separate packages (for Converters and Elude)

build separate packages for percolator, elude and converters

faulty psm statistics without -U flag

without the -U flag, weedOutRedundant() was being invoked at both peptide and psm level! Without the -U flag (by default!) the psm statistics are unreliable.

Percolator: protein level probabilities with Fido

http://students.washington.edu/orserang/fido.shtml

Percoloator: normalized weights print out

Normalized weights vector printed to cerr by Percolator contains only 0s.
Seems to be only a print out problem (nothing appears to be actually broken)

install scripts

substitute installer scripts with CMake directives

serializer.cxx lazy memory release

src/serializer.cxx is lazy in releasing memory and can cause a crash for big xml files (depending on the size of the swap partition and the amount of available RAM). I implemented a temporary fix for sqt2pin, but the solution proposed below by Boris Kolpackov is much more robust.

http://codesynthesis.com/pipermail/xsd-users/2011-June/003316.html

git hooks to generate visual representation of xsd

find out about automatically generating visual (svg) descriptions of xsd files
schemas with .git/hooks

broken gtest

removing redundant variable "base" from CMakeLists causes problems at compile time when -DGTEST="TRUE"

Install procedure do not mention apt-get

The install procedure in
https://github.com/percolator/percolator/blob/master/README
does not mention that you may use apt-get to install the dependencies of percolator. Its clearly an easier path.

RELEASE 2.01

independent versioning for pin and pout schemata

Currently the versioning for pin and pout files is regulated by the
SCHEMA_VERSION_MAJOR and SCHEMA_VERSION_MINOR
variables in the main CMakeLists.
Allow for independent versioning by creating:
_for pin: PIN_VERSION_MAJOR and PIN_VERSION_MINOR
_for pout: POUT_VERSION_MAJOR and POUT_VERSION_MINOR

32bit packages: xsd architecture issue

The 32 bit packages are statically including a 64 bit version of the xsd library. The package installs correctly but any attempt to run percolator will result in the following:
"/usr/bin/percolator: cannot execute binary file"

good practice to set Cmake inputs

Look for standard way of setting input (-D) variables in Cmake
_append command line values rather than overwriting ?

Wrong file permission on installed xml schemes

When I install the deb-packages on my 64bit ubuntu box percolator complains that it cant read the xml-scheme. Checking the file permission I that the file is owned by user tomasoni:

lukask@kall:$ percolator svn/tda/exp/general/worm/perc/10ppm.pin
Percolator version 2.01, Build Date Jun 30 2011 15:09:22
Copyright (c) 2006-9 University of Washington. All rights reserved.
Written by Lukas Käll ([email protected]) in the
Department of Genome Sciences at the University of Washington.
Issued command:
percolator svn/tda/exp/general/worm/perc/10ppm.pin
Started Tue Jul 5 15:04:54 2011
Hyperparameters fdr=0.01, Cpos=0, Cneg=0, maxNiter=10
WARNING: no local xml schema is available to validate the input.
If further errors are encountered, please reinstall Percolator.
:0:0 error: unsupported protocol in URL
svn/tda/exp/general/worm/perc/10ppm.pin:2:255 error: fatal error during schema scan
WARNING: no local xml schema is available to validate the input.
If further errors are encountered, please reinstall Percolator.
:0:0 error: unsupported protocol in URL
svn/tda/exp/general/worm/perc/10ppm.pin:2:255 error: fatal error during schema scan
lukask@kall:$
lukask@kall:~$ ls -al /usr/share/percolator/xml-pin-1-2/percolator_in.xsd
-r-----r-- 1 tomasoni users 13890 2011-06-22 11:46 /usr/share/percolator/xml-pin-1-2/percolator_in.xsd

solveInPlace: failed assertion (HEAD)

....
Merging results from 3 datasets
Selecting pi_0=0.651
Calibrating statistics - calculating q values
percolator: /home/lukask/git/percolator/src/BaseSpline.cpp:378: static
void BaseSpline::solveInPlace(Matrix&, Vector&): Assertion
`isfinite(val)' failed.
Aborted

mailing list

Create mailing list for Percolator users

Sqt2pin Seg fault

When executing the fms2 file
/ext/behemot_public/ms2/mattia/08183.ms2 I get a seg-fault. Is it due to absent EZ lines?

RELEASE 2.0

Release version 2.0

error message on xsd schema not found

This is not very informative

error: fatal error during schema scan
:0:0 error: unable to resolve host/address 'per-colator.com'

noise in percolator tab output

automate release process

additional pi_0 system level test

Add system level test that ensures that pi_0 for test files remains within 5% from its current value

stdout of sqt2pin

sqt2pin -o fname generates xml to stdout. Please generate xml to output only when there are no -o option given

low pi_0 estimation causes assert(q1>median) to fail

Low pi_0 estimation causes assert(q1>median) to fail during percolator's normalization step

[from Torsten Ueckert's email - 8th March 2011 ]

I get an exit code 255 from percolator.exe wile processing the
attached input file.

This is data from a Mascot search with the OR20070924_S_mix7_06.RAW file.

Percolator astray

for "some" input, Percolator gives the warning:
"Reseting score vector, using default vector".
The weights calculated for XCorr and/or deltaCn are negative

See the five subfolders of:
dirac.scilifelab.se:/scratch/temp/vagisha_percolatorAstray/yrc_5expts_w_perc_wrng_after_tide_fix/

Sqt2pin: unique name for Tokyocabinet temp file

Sqt2pin.cpp line 112: tokyoCabinetTmpFN = "/tmp/percolator-tmp.tcb" causes problems when more then one instance of Sqt2pin is running.

make percolator-converters dependent on percolator

; lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 10.10
Release: 10.10
Codename: maverick

$ sudo dpkg --force-all --install percolator-converters-v2_00-linux-amd64.deb 
(Reading database ... 315483 files and directories currently installed.)
Unpacking percolator-converters (from percolator-converters-v2_00-linux-amd64.deb) ...
dpkg: warning: overriding problem because --force enabled:
 trying to overwrite '/usr/share/percolator/xml-pin-1-2/percolator_in.xsd', which is also in package percolator 2.00.0
Setting up percolator-converters (2.00.0) ...

issue cross-compiling 32bit on 64bit with cmake

Cpack's option TARGET_ARCH does not influence Cmake's compiling choices.
The 32bit packages produced are broken: I removed them from https://github.com/percolator/percolator/downloads

There exists a CMAKE_CXX_FLAGS="-m32" option. Investigate.

Processing messages from sqt2pin are confusing

I get the following processing info on stderr:

Sqt2Pin version 2.01, Build Date Jun 30 2011 12:52:26
Copyright (c) 2010 Lukas Käll. All rights reserved.
Written by Lukas Käll ([email protected]) in the
Department of Biochemistry and Biophysics at the Stockholm University.
Issued command:
sqt2pin ../target200ppm/Celegans_Jupter_C12_120min_grad.sqt ../reverse200ppm/Celegans_Jupter_C12_120min_grad.sqt

Reading from sqt file(s) and writing to database(s):
reading ../target200ppm/Celegans_Jupter_C12_120min_grad.sqt
reading ../reverse200ppm/Celegans_Jupter_C12_120min_grad.sqt

Writing database(s) to pin file:
writing ../target200ppm/Celegans_Jupter_C12_120min_grad.sqt (and correspondent decoy file)

Termination successful.

1. write sqt2pin and not Sqt2pin
2. We are hopefully not writing to the sqt file writing ../target200ppm/Celegans_Jupter_C12_120min_grad.sqt (and correspondent decoy file) -- instead we are writing the contense from those files to stdout
3. What does Termination successful mean?

update percolator_out.xsd to handle unimod ptms

Current output generated from files with ptms does not validate against percolator_out.xsd

/tmp/PERCOLATOR_psm_ptms.pout.xml:1509: element peptide_seq: Schemas validity error : Element '{http://per-colator.com/percolator_out/13}peptide_seq', attribute 'seq': [facet 'pattern'] The value 'S[UNIMOD:21]DKYSKLK' is not accepted by the pattern '([A-Z][#*@]?)+'.

unimod ids for PTMs

The current PTM representation is sequest style * @ etc...
To be changed to unimod id's (eg MY[unimod:1]PEPTIDE)

Percolator: input error under win (v 1.17) (Carmen, Thermo)

Sqt2pin: psm grouping

Current psm grouping (by scan number) seems to cause information from different files to be retained in memory longer than necessary. Consider instead splitting the fragSpectrumScan: scanNumber="[scan #]_[Filename]".

[from Barbara's email, 7th March 2011]

I noticed that the schema groups psms first by scan number. So within the
tag, there are multiple PSMs all for spectra with the
same scan number but with different charge states and from different files.
For example (I've removed some of the attributes)

<fragSpectrumScan experimentalMassToCharge="912.6508" scanNumber="35">
  <peptideSpectrumMatch  id="FileName_35_2_1">
    <features>  ...   </features>
    <peptide>   ...   </peptide>
  </peptideSpectrumMatch>

  <peptideSpectrumMatch  id="FileName_35_4_1">
    <features>  ...   </features>
    <peptide>   ...   </peptide>
  </peptideSpectrumMatch>

  <peptideSpectrumMatch  id="DifferentFileName_35_2_1">
    <features>  ...   </features>
    <peptide>   ...   </peptide>
  </peptideSpectrumMatch>
</fragSpectrumScan>

Seems like this would require that all the files be read in and stored
before the pin.xml can be written out. Perhaps percolator requires that the
psms be grouped in this way, but if not maybe it's possible to change the
schema slightly so that the pin.xml file is written as the .sqt files are
read so that there's no need to store much in memory.

percolator stdin

I am missing an option to read pin files from stdin. Ot would be nice to execute a pipe structure such as "sqt2pin target.sqt decoy.sqt | percolator -E - -p"

Sqt2Pin: error on large input (v.1.7) (Genn, UoW)

error message when percolator is invoked without parameters

suggest --help instead of

percolator: /scratch/temp/percolator/src/Caller.cpp:602: void Caller::readFiles(): Assertion `false' failed.
Aborted

tokyocabinet on windows

converters on Windows: kyotocabinet (under gpl)? or (better) tcport (http://code.google.com/p/tcport/)? Would allow to build and debug Percolator directly in Windows.

Need for a system test checking the consistency of the pout files

The xml-output from percolator should be checked against its xsd. If nowhere else in a system test. It would be good to also test percolator output given input files with ptm's.

new pin-1-2

pin: experimentalMassToCharge element is duplicated in fragSpectrumScan element and psm element. Remove it from fragSpectrumScan

<exp_mass> and <calc_mass> missing from pout

Scores.cpp line 75.
Output <exp_mass> and <calc_mass> whenever the information is available; boolean MassHandler::monoisotopic connected to old -M option is no longer meaningful.

BOOST routines instead of Array Library?

consider substituting Array Library with BOOST routines

Documentation example inconsistent with actual usage

The official documentation on:

http://per-colator.com/example/

reads:

percolator -E pin.xml -X pout.xml > yeast-01.psms

Which fails:

ERROR: percolator was invoked with an invalid option.
Please run "percolator --help."

The new syntax should be stated there on the example instead:

percolator [-X pout.xml] [other options] pin.xml

Deprecated options

The options -e -c -y -o, -s, -Y, -2 of percolator 1.15 seem redundant - please remove or explain better.

percolator: ENZYME

The kind of enzyme used DOES NOT INFLUENCE THE OUTPUT

Percolator performance issue with unique peptides

Percolator takes 10-50x longer than SEQUEST if we use the peptide level q-values -- PSM

Produce xml output for Percolator through Xerces

Use class src/Serializer (FragmentSpectrumScanDatabase.cpp line 130)

report all pi_0s (psm, peptide and protein) in pout

additional psm and peptides system level tests

check pep and q-value of 1st, 50th and 100th position is within 5% from what it currently is for both psm and peptides

report on vm configurations

Write a report with as many details as possible about the virtual machines you have for cross-compiling and testing.

stl::unique in weedOutRedundant()

consider using stl::unique in Scores::weedOutRedundant()