mljs / spectra-processing

Various methods to help spectra processing

ml-spectra-processing

Various functions to process spectra. Those are pure functions.

They are sorted in various categories:

• matrix: functions on ml-matrix instances (or array or array)
• x: functions that apply on a 1D array
• xy: functions that apply on an object {x: [], y:[]}
• xy2: functions that apply on an array of array of 2 numbers [[x,y], [x,y]]
• xyArray: functions that apply on an array of objects {x: [], y:[]}
• xyObject: functions that apply on an array of point [{x,y}]
• xreim: functions that apply on an object {x: [], re:[], im:[]}
• reim: functions that apply on an object {re: [], im:[]}
• zone: functions that apply on a zone {from,to}
• zones: functions that apply on an array of zone [{from,to}]

The name of the functions start with the first argument on which they apply.

Installation

$ npm i ml-spectra-processing

Usage

import { xAdd } from 'ml-spectra-processing';

const array = [1, 2, 3];
const result = xAdd(array, 5);
console.log(result);

API Documentation

License

MIT

spectra-processing's Issues

there are functions that only accepts number[]

Documentation of parameters

We are using typedoc to generate the documentation in typescript projects.

When describing a parameter the - is mandatory otherwise the description does not appear in the documentation.

You can always check the documentation locally using:

npm i --global typedoc
typedoc src/index.ts

Example of modified code:

8600c20

Example of resulting help

@wadjih-bencheikh18 Please could you have a look through all the code to fix similar problems ?

feat: add matrixBoxPlot

Should accept an array of NumberArray and returns an object:

   {
    q1: [],
    median: [],
    q3: [],
    min: [],
    max: [],
  };

Add noise calculation for matrix

• add matrixMedian
• add matrixSanPlot

replace new Array by new Float64Array

If possible we should replace new Array by new Float64Array and adapt the tests.

Please note that new Float64Array fill by default the array by '0' !!!! No need to fill them.

xSampling removes points from the end of the array

@lpatiny In a special case, downSampling method removes points from the end when the number of points doesn't evenly divide the array length. This significantly impacts the Hilbert transform's result when using the forceFFT option.

Tests should use directly index

In order to ensure that the method is really exposed by the package we should directly use the method from index.ts

Check if gaussian

@aiday-mar could you add a test that actually checks that it is a gaussian distribution with the correct SD ?

https://github.com/mljs/spectra-processing/blob/adbf2b4b06cf6f25575c87e87926d7b4cbb2e3c7/src/utils/__tests__/createRandomArray.test.ts

I changed the code so that it is seeded random number to avoid problems with the test.

You could create an histogram (with xHistogram) and use as developper dependency http://mljs.github.io/spectra-fitting/ to check if it match a gaussian.

Add phase correction in reim

Add a function allowing phase correction. What is the best name for this mathematical operation ? Phase correction seems quite NMR specific

No inline help for parameters

I don't know if it is an isolated case but we don't need a help for the property of a parameter that has a specific interface. The help is at the level of the definition of the interface.

Also if we don't have a description of the parameter there is no reason to add an @param

4c6b59a

@wadjih-bencheikh18 could you have a look if there are similar problems ?

Create an index per folder

avoid fallback to number in xyIntegral

spectra-processing/src/xy/xyIntegral.ts

Line 44 in c39ed5d

if (x === undefined || y === undefined || x.length < 2) return 0;

Similar to 5f9aae3

version 12.4.0 can not directly used.

I got this error at the moment to install the last version.

Improve createXArray

The parameters are confusing and not intuitive

https://github.com/mljs/spectra-processing/blob/master/src/utils/createXArray.ts

We should probably create more than one method. One with step and the other one with fill. I don't see the point with step to ignore from or to.

This is related to #124

No idea for the names. @aiday-mar a proposal ?

Move random utilities to `ml-random`

https://github.com/mljs/random

add matrixSetSubMatrix

Split method xSequentialFill in xSequentialFillFromTo and xSequentialFillFromStep

We try to have too many options in this method and I think it should be split in 2 methods. The current code is pretty complex.

spectra-processing/src/x/xSequentialFill.ts

Lines 23 to 26 in 220f6ce

	export function xSequentialFill(
	input: NumberArray | undefined,
	options?: XSequentialFillOptions,
	): NumberArray;

xSequentialFillFromTo(from, to, size, {ArrayConstructor})

xSequentialFillFromStep(from, step, size, {ArrayConstructor})

Both methods will return a Float64Array or the output if it has been provided.

Check return value type

If a method returns for sure a Float64Array we should not say it returns a DoubleArray but specify it is a Float64Array

export function xSubtract(
  array1: DoubleArray,
  array2: DoubleArray | number,
): DoubleArray {

Hilbert-transform could use FFT for non power of 2 data

In case the data is not a power of 2 Hilbert transform could use xSampling but this should be an option. I don't think that it will be 100% the same result if you go through power of 2 (to be checked).

If it is not 100% please add an option 'forceFFT' that is only useful if it is not a power of 2 and we use xSampling.

Add test for numberSort

Seems there are no tests for this function

https://github.com/mljs/spectra-processing/blob/master/src/utils/numberSort.ts

PQN should use median

utils/createRandomXArray()

Get inspired by https://stackoverflow.com/questions/14058340/adding-noise-to-a-signal-in-python

We should be able to create an array containing random number

As testcase we could use xNoiseStandardDeviation.

@aiday-mar Could you propose an API and options for this new function ?

box plot and outliers

Need to calculate interquartile distance

https://rpkgs.datanovia.com/rstatix/reference/outliers.html#:~:text=Boxplots%20are%20a%20popular%20and,points%20(or%20extreme%20outliers).

https://www.statology.org/how-to-read-box-plot-with-outliers/

new function xyMassCenter

Get inspired by:

• https://github.com/mljs/spectra-processing/blob/main/src/x/xMeanWeighted.ts

Name the new function xyMassCenter

Don't forget to add the export in xy/index.ts

Add in the previous method @deprecated Please use xyMassCenter

Add step in createXArray

If step parameter is given we would ignore 'to'

Typedoc and options

In order to be able to describe the documentation of optional parameters with their default value we should either describe them 'inline' or create a new type in which we take the name of the method and append 'Options'. Like for example: XFindClosestIndexOptions.

If you have only 1 to 3 options it is possible to describe them inline but otherwise the new type is required. Same is true when the same type is reused in many functions.

Here is an example of inline help:

63f3fc9

And the corresponding result in which we see the default value

Please could you go through all the code to fix similar problems ?

Rename "monotone" to "monotonic"

• Change isMonotone to isMonotonic and make it return a number:
  • 0 -> not monotonic
  • 1 -> monotonic increasing
  • -1 -> monotonic decreasing
  • For a constant function, I guess we can decide to return 1?
• Remove isMonotoneIncreasing function

The exported function should be renamed to xHilbertTransform

@lpatiny I made a mistake

spectra-processing/src/x/xHilbertTransform.ts

Line 6 in 3c6b2d2

export function hilbertTransform(

Expose the different processing utilities per folder

Add test to reimAutoPhaseCorrection

One of the only code not being tested is reimAutoPhaseCorrection

@jobo322 could you add a test ?

Dead code or bug in xNoiseSanPlot

The variables assigned here are not used after the assignment:

spectra-processing/src/x/xNoiseSanPlot.ts

Lines 160 to 161 in 5cd995e

	initialNoiseLevelPositive = initialNoiseLevelPositive / correctionFactor;
	initialNoiseLevelNegative = initialNoiseLevelNegative / correctionFactor;

/cc @lpatiny @wadjih-bencheikh18 who worked on this code.

feat: add matrixAutoCorrelation

We can get inspired by https://github.com/cheminfo/spectra-processor/blob/55b53ce69f6cc78984880bf76d7e7beb72ac4c22/src/spectra/getAutocorrelation.js#L7-L27

Add xSampling

xSampling(array, options)

options:

• distribution: 'uniform', 'log'
• nbPoints: default 10
• firstPoint: true / false
• endPoint: true / false

Other ideas:

https://numpy.org/doc/stable/reference/generated/numpy.logspace.html
https://numpy.org/doc/stable/reference/generated/numpy.linspace.html

Remove Uint16Array

There is probably no reason some methods can accept specifically Uint16Array. Either we accept any array of number of we only accept DoubleArray.

I just added in cheminfo-types the type NumberArray. We should now check and replace the code where Uint16Array is specified to check if it can be replaced by NumberArray.

https://github.com/cheminfo/cheminfo-types/blob/main/src/core/NumberArray.d.ts

Problem building the documentation

With the default action to build the documentation I have the following error:

https://github.com/mljs/spectra-processing/runs/2499211793?check_suite_focus=true#step:3:4

In order to let it work I had to add the following 2 lines

2c2cbee

What am I doing wrong ? I would expect that the default action works.

Accept Matrix in the matrices functions

Improve reimAutoPhaseCorrection by removing outliers

Test failing

@jobo322 Could you check the problem of the failing tests ?

https://travis-ci.com/github/mljs/spectra-processing/jobs/396368547#L367-L375

Add options

In:

https://github.com/mljs/spectra-processing/blob/master/src/x/xMaxIndex.ts
https://github.com/mljs/spectra-processing/blob/master/src/x/xMinIndex.ts

Allow to have the from / to / fromIndex / toIndex like in:

https://github.com/mljs/spectra-processing/blob/master/src/x/xMaxValue.ts

What you would need to do:

• fork the project on your account
• clone the project from your account
• try to add the options
• add test cases
• create a pull request

This is a typescript project and is therefore a little bit different from javascript but I think you could manage.

After cloning the project don't forget:

• npm i
• npx jest --watch

Migrate project to typescript

Before you start please have a loot at:

• https://github.com/cheminfo/generator-cheminfo/blob/main/START.md
• https://github.com/cheminfo/generator-cheminfo/blob/main/ts-migration.md