Debian packagin is broken

relies on stdeb thus setup.py
--> use wheel2deb

[BM29] Plugin for HPLC experiment

[BM29] recurent H5py errors when saving Rambo-Tainer invariants in subtracted file

[ID02] Refactor plugins from ID02 into a package

This will make code simpler to maintain on the one hand and allow the inclusion of XPCS in the future.

id02.singledetector: missing metadata from C216

id02.singledetector: dark averaging/correction is missing

uncloded HDF5 file in metadata plugin for ID02

Dummy value not passed to azimuthal integrator

[BM29] Plugin for ab-initio modeling

[BM29] std based on variance in azimuthal space

Would easily allow to spot scaling issues in Poisson noise model

[BM29] integration of processed data into the data-portal

save resulting dataset in a separated folder
save input and output JSON into the folder
save plot to feed the gallery
call the data-injector

[BM29] Use worker instead of azimuthal integrators

Especially for caching

[BM29] subtract plugin should have a symbolic link to the NXinstrument of the sample file

[ID02] Make output files Nexus compliants

Default representation at the entry level
Correct the NXdata for automatic view in silx (with errors @uncertainties attribute)
Flag NXProcess: date, plugin-date, plugin-version,
Flag HS32{CFNVZ} with interpretation=spectrum
Rename I0 and I1 as UnCorr

switch to the same schem as silx

implies many re-ordering of code

handle the case where dark/flat/spline are not the good ones ...

use binning (actually an average) of those data

[BM29] memcached curve transfer

[BM29] Correct formula for standard error of mean

For this:

record sum_signal, sum_norm and sum(ω'²) into HDF5 at the integration level.
export sigma = sqrt(sum(sum_signal)/sum(sum_norm)) possibly mulitplied by sqrt(sum(ω'²)) (to get the SEM instead of the STD)

id02.singledetector: filename for corrected images is missing number/extension

[ID02] XPCS plugin should increment entries in Nexus files

json.JSONEncoder does not have an "encoding" argument

PyTango.DevFailed: DevFailed[
DevError[
    desc = TypeError: __init__() got an unexpected keyword argument 'encoding'
           
  origin = Traceback (most recent call last):
  File "/users/denolf/virtualenvs/dahu/ubuntu_20_04/lib/python3.8/site-packages/dahu/server.py", line 263, in getJobOutput
    return Job.getDataOutputFromId(jobId, as_JSON=True)
  File "/users/denolf/virtualenvs/dahu/ubuntu_20_04/lib/python3.8/site-packages/dahu/job.py", line 493, in getDataOutputFromId
    output = json.dumps(job._output_data, skipkeys=True, allow_nan=True,
  File "/users/denolf/virtualenvs/dahu/ubuntu_20_04/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
TypeError: __init__() got an unexpected keyword argument 'encoding'

  reason = PyDs_PythonError
severity = ERR]

DevError[
    desc = Failed to execute command_inout on device id00/dahu/1, command getJobOutput
  origin = Connection::command_inout()
  reason = API_CommandFailed
severity = ERR]
]

[ID02] File produced by Eiger2 not compatible

Error ERROR in SingleDetector: Expected ONE collections is expected in
entry, got 0 in /data/id02/inhouse/com/20200309//Test_eiger2_00007_raw.h5
<HDF5 group "/entry_0000" (5 members)> while calling SingleDetector.process
<type 'exceptions.RuntimeError'>: ERROR in SingleDetector: Expected ONE
collections is expected in entry, got 0 in
/data/id02/inhouse/com/20200309//Test_eiger2_00007_raw.h5
<HDF5 group "/entry_0000" (5 members)>
  File "/usr/lib/python2.7/dist-packages/dahu/job.py", line 214, in _run_
method()
  File "/usr/lib/python2.7/dist-packages/dahu/plugins/id02.py", line 657,
in process
self.metadata = self.parse_image_file()
  File "/usr/lib/python2.7/dist-packages/dahu/plugins/id02.py", line 902,
in parse_image_file
(len(collections), self.image_file, self.entry))
  File "/usr/lib/python2.7/dist-packages/dahu/plugin.py", line 108, in
log_error
raise RuntimeError(err)

[ID02] Offer compression for datasets with interpretation "image"

There are 4 compression schemes: None, gzip, LZ4 and bitshuffle(with LZ4)

metadata got lost

From: Peter Boesecke [email protected]
To: [email protected]
Subject: dahu scaler file nj17_scalers_00000.h5
Date: Fri, 17 Oct 2014 17:58:08 +0200

Dear Jerome,

after Laurent and Sebastien have solved the server problem, I could test
again your dahu:

dir: /mntdirect/_data_opid02_inhouse/com/20141008/data3
file: nj17_scalers_00000.h5

A lot of MCS values are not written any more. You can check your
corrections on monday on ID02 using the frelon or saxs detector.

If you need help to start an acquisition ask Sylvain Prevost (40-29) or
anybody else from ID02.

Example:

spec session: frelon ( session saxs forbeamline users)

The basic commands are:

ccdactive frelon
ccdnewfile
acquisitions work with:
ccdacq 0.1 "test"
or
id02limamvdc 10 0.006 0.7 1.01 0.1 1.03 1 11 "test"

Greetings

Peter

Dr. Peter Boesecke
ID01 & ID02
European Synchrotron Radiation Facility (ESRF)
CS 40220
F-38043 GRENOBLE CEDEX 9

Tel. +33 (0) 4 7688 2400
Fax. +33 (0) 4 7688 2542

Email [email protected]

[BM29] Plugin for auto-subtraction and plot generation

[BM29] put sample name in title of the plot

[BM29] propagate timestamps of frames

allow absolute or relative timestamps.

[ID02] XPCS what to do if some bins have no contributions

Example: 22 q-bins requested but the 16 first fall into the mask, only 6 q-bins are populated ?

Shouln't the arrays be populated with 0 (or Nan) ?

@pierrepaleo

id02.metadata

skip entry/hdf5_filename/plugin_name from Information group

[ID02] Error propagation

Initialize variance array from user input using numexpr
Propagate errors (as standard deviation, i.e. sqrt of variance)

[BM29] export_ascii should work on subtracted files

[ID02] convert plugins from Python2 to Python3

[core] Make the DataCache class a borg or not...

Sometimes it could be convieniant to have this DataCache object not being a singleton.

related to the memory-leak from ID02

The memory occupation after a while is 40G, the dict with 13522 finished jobs represents 40M (5k/job, mainly path to saved files), the geometry cache has been emptied without without noticeable freeing of memory. There is no memory leak on the GPU (789M occupied)
--> Valgrind ?

Documentation is broken

still relies on setup.py

kif / dahu Goto Github PK

dahu's People

Contributors

Stargazers

Watchers

Forkers

dahu's Issues

Peter

Email [email protected]

Recommend Projects

Recommend Topics

Recommend Org