openptv / postptv

Post processing routines for analysing PTV data.

Flowtracks - postprocessing of 3D-PTV data

This package contains Flowtracks, a Python-package for post-processing of 3D Particle Tracking Velocimetry particle/trajectory databases.

The full documentation for this package may be built from the Sphinx sources in the doc/ directory. It is also available online:

http://flowtracks.readthedocs.org

Please refer to that documentation for the full information on installing, reference documentation and usage examples contained in the package.

The program is distributed under the terms of the GNU General Public License, version 3.0. For details, see the LICENSE.txt file.

How to cite this work

Meller, Y and Liberzon, A 2016 Particle Data Management Software for 3DParticle Tracking Velocimetry and Related Applications – The Flowtracks Package. Journal of Open Research Software, 4: e23, DOI: http://dx.doi.org/10.5334/jors.101

Getting Started

Obtaining the package and its dependencies

The most recent version of this package may be found under the auspices of the OpenPTV project, in its Github repository,

https://github.com/OpenPTV/postptv

Dependencies:

• The software depends on the SciPy package, obtainable from http://www.scipy.org/

• Some features depend on the Matplotlib package. Users which need those features may get Matplotlib at http://matplotlib.org/

Installation

To install this package, follow the standard procedure for installing Python modules. The package may be installed either systemwide, in the default location, or for a single user, by changing the install location and updating environment variables in the standard way, as indicated by the Python documentation [1].

For a default systemwide installation: using a terminal, change directory into the root directory of theis program's source code, then run

pip install -r requirements.txt

Note that for the default install you may need administrative privileges on the machine you are using. Consult the Python documentation for the single-user install procedure.

The install script will install the Python package in the default place for your platform. Additionally, it will install example scripts in a subdirectory flowtracks-examples/ under the default executable location, and a the documentation in the default package root. For information on where these directories are on your platform (and how to change them), refer to the Python documentation [1]. Other standard features of the setup script are also described therein.

The examples are Jupyter notebooks [2], and can be previewed without any special setup under the examples section of

http://flowtracks.readthedocs.org

[1] Python documentation: https://docs.python.org/2/install/index.html [2] http://jupyter.org/

postptv's Issues

question: why in pairs segs=True?

Hi @yosefm, just a question, why when we look at pairs in some frame, we request for trajectory to be active also in the next frame?

postptv/flowtracks/pairs.py

Line 57 in 81f27cd

segs=True)

bug in io.trajectories_table?

i've found that flowtracks.io.trajectories_table has been using older PyTables convention
http://www.pytables.org/MIGRATING_TO_3.x.html

e.g. alexlib@5cf008c

Inverse weighting is not implemented?

The example notebook in /examples https://github.com/OpenPTV/postptv/blob/master/examples/hdf5_scene_analysis.ipynb

apparently uses inverse weighting. but I get NotImplementedError. Looking into the file interpolation.py I do see that it's not implemented.

https://github.com/OpenPTV/postptv/blob/master/flowtracks/interpolation.py#L703

Shall we fix the example or complete the inverse weighting implementation?

izip used in scene.py but not compatible with Python 3.x

In scene.py it.izip is used in line 34 and 457. This should be changed from "it.izip()" to just "zip()" to be compatible with python 3.x

python 3 support

I guess we have urgently move to python 3
@yosefm could you please help us with the direction?

[feature request] linking of the broken trajectories

One of the common issues that can be solved through post-processing is the linking of the broken trajectories. There are publications by Beat Luethi (called glueing) and Haitao Xu (multi-dimensional tracking) that dealt with this problem for 3D-PTV specifically and there is an old Matlab code that I've wrote some time ago

Matlab linking code

The idea is rather simple - for each two trajectories that are possibly broken pieces of the same trajectory project the edges (last for the earlier and first for the later) towards each other and see if they are within the acceptable tolerance (of distance, velocity and/or acceleration and time difference). If they are, connect, replace them by a single peace, make it smooth again, repeat the search.

bug in inverse weigthing

AttributeError                            Traceback (most recent call last)
Cell In[3], line 12
      7 # Here we start to use the repeated-interpolation machinery,
      8 # By informing the interpolant of the current frame data,
      9 # and then querying it about that data without having to repeat it.
     10 interp.set_scene(frame.tracers.pos(), frame.particles.pos(), 
     11     frame.tracers.velocity())
---> 12 neighb_base = interp.which_neighbours()
     14 # Check that we have a particle with the desired number of tracers.
     15 candidates = neighb_base.sum(axis=1) >= 10

AttributeError: 'InverseDistanceWeighter' object has no attribute 'which_neighbours'

when running repeated_interpolation example

numpy changed the indexing and it causes errors

I think we need to change the way we use indexing in trajectory.py

postptv/flowtracks/trajectory.py

Line 68 in 81f27cd

return self.__dict__[attr][selector]

from

self.__dict__[attr][selector]

to

self.__dict__[attr][np.array(selector).astype(np.int)]

for instance particle_pairs fail for the call of the type:

trajid = [0,1,2]
particle_pairs(original,shifted,trajids=np.array(trajid),time_points=np.zeros(3))

with the error

/Users/alex/miniconda3/envs/postptv/lib/python2.7/site-packages/flowtracks/pairs.pyc in particle_pairs(primary_trajects, secondary_trajects, trajids, time_points)
     52         prim_in_frame_ids = np.unique(trajids[coord_locator])
     53         prim_in_frame = [t for t in prim_traj if t.trajid() in prim_in_frame_ids]
---> 54         prim_parts = take_snapshot(prim_in_frame, frame_num, schema)
     55 
     56         sec_in_frame_ixs = trajectories_in_frame(secondary_trajects, frame_num,

/Users/alex/miniconda3/envs/postptv/lib/python2.7/site-packages/flowtracks/trajectory.py in take_snapshot(trajects, frame, schema)
    305         first_frame = traj.time()[0]
    306         for prop in copy_keys:
--> 307             kwds[prop][trix] = traj.__dict__[prop](frame - first_frame)
    308         kwds['trajid'][trix] = traj.trajid()
    309 

/Users/alex/miniconda3/envs/postptv/lib/python2.7/site-packages/flowtracks/trajectory.py in getter(self, selector)
     66                 return self.__dict__[attr]
     67             else:
---> 68                 return self.__dict__[attr][selector]
     69 
     70         def setter(self, new_val, selector=None):

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

probably also in other places, e.g.

postptv/flowtracks/trajectory.py

Line 74 in 81f27cd

self.__dict__[attr][selector] = new_val

tests fail on Python 3 with the numpy 1.18.3

`(test) MacBook-Pro:postptv alexliberzon$ pytest tests/
============================= test session starts ==============================
platform darwin -- Python 3.7.7, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /Users/alexliberzon/Downloads/flowtracks/postptv
collected 25 items

tests/test_analysis.py .. [ 8%]
tests/test_interp.py ........F. [ 48%]
tests/test_io.py .... [ 64%]
tests/test_scene.py ....... [ 92%]
tests/test_sequence.py .. [100%]

=================================== FAILURES ===================================
____________________________ TestJacobian.test_inv _____________________________

self = <test_interp.TestJacobian testMethod=test_inv>

def test_inv(self):
    pos = np.array([[0., 0., 0.]])
    tracer_pos = np.array([
        [0.001, 0, 0],
        [-0.001, 0, 0],
        [0,  0.001, 0],
        [0, -0.001, 0],
        [0, 0,  0.001],
        [0, 0, -0.001]
    ])
    # Basically we interpolate something based on the average position
    # change, because it's easy for me to visualize.
    interp_data = tracer_pos*2

    interp = interpolation.interpolant('inv', 6, 3)
    interp.set_scene(tracer_pos, pos, interp_data)

    local = interp.interpolate()
    np.testing.assert_array_equal(local, np.zeros((1, 3)))

    jac = interp.eulerian_jacobian()

  self.assertTrue(np.all(jac[:, [0, 1, 2], [0, 1, 2]] != 0))

E IndexError: index 1 is out of bounds for axis 1 with size 1

tests/test_interp.py:170: IndexError
=============================== warnings summary ===============================
tests/test_io.py::TestPtvis::test_trajectories_hdf
tests/test_io.py::TestPtvis::test_trim_hdf
/Users/alexliberzon/opt/miniconda3/envs/test/lib/python3.7/site-packages/flowtracks/io.py:710: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
dtype = np.dtype(fields)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
=========================== short test summary info ============================
FAILED tests/test_interp.py::TestJacobian::test_inv - IndexError: index 1 is ...
=================== 1 failed, 24 passed, 2 warnings in 4.53s ===================
`

trajectory length

Hello Yosef and Alex,
Thank you for the valuable package you provided.
I am using flowtracks for Lagrangian studies and I have two questions I hope you can help me with those:

1. I get the length of each trajectory by using following line:
   traj.pos()[-1,0]-traj.pos()[0,0]
   the maximum length I get is 0.16 but when I plot traj positions I see the maximum length of the trajectories is equal to the length of the observation volume (0.3 m).
   my guess is that the plot function just join small trajectories and make a longer one or so. If yes, based on what criteria it does that? Please correct me if I am wrong.
   2)is there any class for Lyapunov exponent calculation in flowtracks?
   Thank you for all the help,

suggest to create a gallery of postptv scripts and Jupyter notebooks

This could be a necessarily trigger for us to do everything in postpv. I think as a first thing a gallery of Jupyter notebooks with some demos could be a good option.