kif / dahu Goto Github PK
View Code? Open in Web Editor NEWSoftware chunks for ID02 upgrade program
License: MIT License
Software chunks for ID02 upgrade program
License: MIT License
Include
Sometimes it could be convieniant to have this DataCache object not being a singleton.
related to the memory-leak from ID02
Or should I build it to browse it ?
Error ERROR in SingleDetector: Expected ONE collections is expected in
entry, got 0 in /data/id02/inhouse/com/20200309//Test_eiger2_00007_raw.h5
<HDF5 group "/entry_0000" (5 members)> while calling SingleDetector.process
<type 'exceptions.RuntimeError'>: ERROR in SingleDetector: Expected ONE
collections is expected in entry, got 0 in
/data/id02/inhouse/com/20200309//Test_eiger2_00007_raw.h5
<HDF5 group "/entry_0000" (5 members)>
File "/usr/lib/python2.7/dist-packages/dahu/job.py", line 214, in _run_
method()
File "/usr/lib/python2.7/dist-packages/dahu/plugins/id02.py", line 657,
in process
self.metadata = self.parse_image_file()
File "/usr/lib/python2.7/dist-packages/dahu/plugins/id02.py", line 902,
in parse_image_file
(len(collections), self.image_file, self.entry))
File "/usr/lib/python2.7/dist-packages/dahu/plugin.py", line 108, in
log_error
raise RuntimeError(err)
This will make code simpler to maintain on the one hand and allow the inclusion of XPCS in the future.
PyTango.DevFailed: DevFailed[
DevError[
desc = TypeError: __init__() got an unexpected keyword argument 'encoding'
origin = Traceback (most recent call last):
File "/users/denolf/virtualenvs/dahu/ubuntu_20_04/lib/python3.8/site-packages/dahu/server.py", line 263, in getJobOutput
return Job.getDataOutputFromId(jobId, as_JSON=True)
File "/users/denolf/virtualenvs/dahu/ubuntu_20_04/lib/python3.8/site-packages/dahu/job.py", line 493, in getDataOutputFromId
output = json.dumps(job._output_data, skipkeys=True, allow_nan=True,
File "/users/denolf/virtualenvs/dahu/ubuntu_20_04/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
TypeError: __init__() got an unexpected keyword argument 'encoding'
reason = PyDs_PythonError
severity = ERR]
DevError[
desc = Failed to execute command_inout on device id00/dahu/1, command getJobOutput
origin = Connection::command_inout()
reason = API_CommandFailed
severity = ERR]
]
Especially for caching
does cProfile work within a thread ?
use binning (actually an average) of those data
relies on stdeb thus setup.py
--> use wheel2deb
Example: 22 q-bins requested but the 16 first fall into the mask, only 6 q-bins are populated ?
Shouln't the arrays be populated with 0 (or Nan) ?
Dahu still relies on numpy.distutils
For this:
sigma = sqrt(sum(sum_signal)/sum(sum_norm))
possibly mulitplied by sqrt(sum(ω'²)) (to get the SEM instead of the STD)Would easily allow to spot scaling issues in Poisson noise model
still relies on setup.py
From: Peter Boesecke [email protected]
To: [email protected]
Subject: dahu scaler file nj17_scalers_00000.h5
Date: Fri, 17 Oct 2014 17:58:08 +0200
Dear Jerome,
after Laurent and Sebastien have solved the server problem, I could test
again your dahu:
dir: /mntdirect/_data_opid02_inhouse/com/20141008/data3
file: nj17_scalers_00000.h5
A lot of MCS values are not written any more. You can check your
corrections on monday on ID02 using the frelon or saxs detector.
If you need help to start an acquisition ask Sylvain Prevost (40-29) or
anybody else from ID02.
Example:
spec session: frelon ( session saxs forbeamline users)
The basic commands are:
ccdactive frelon
ccdnewfile
acquisitions work with:
ccdacq 0.1 "test"
or
id02limamvdc 10 0.006 0.7 1.01 0.1 1.03 1 11 "test"
Greetings
Dr. Peter Boesecke
ID01 & ID02
European Synchrotron Radiation Facility (ESRF)
CS 40220
F-38043 GRENOBLE CEDEX 9
Tel. +33 (0) 4 7688 2400
Fax. +33 (0) 4 7688 2542
skip entry/hdf5_filename/plugin_name from Information group
implies many re-ordering of code
There are 4 compression schemes: None, gzip, LZ4 and bitshuffle(with LZ4)
allow absolute or relative timestamps.
The memory occupation after a while is 40G, the dict with 13522 finished jobs represents 40M (5k/job, mainly path to saved files), the geometry cache has been emptied without without noticeable freeing of memory. There is no memory leak on the GPU (789M occupied)
--> Valgrind ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.