allenneuraldynamics / spim-core

common features shared across Allen Institute SPIM systems

SPIM Core

a consolidation of core features across Allen Institute SPIM systems

Installation

This package can be installed from Pypi with pip install spim-core. To install this package from the github repository in editable mode, from this directory invoke pip install -e .

spim-core's Issues

schema logs should go to their own files

We need to add another handler to main.py to do this.

Enable generic component locating

We need a consolidated way to orient various components in space relative to each other without having to deal with flipping coordinate signs, etc in code. Locations of components can be specified in the config.

We should use ROS' transformations library, which is self-contained, and contains general utilities for rotation, scaling, etc.

Aside, but relevant: ROS supports a way of exporting kinematic trees from Solidworks with a "URDF file exporter". This would be a great way to specify the locations of objects without needing to bookkeep the file manually.

Folder / File structure and naming conventions

Dataset folder name should be: <microscope_name>_<specimin id>_<year>-<month>-<day>_<hour>-<min>-<sec>

Imaging folder should change from micr to the name of the microscope (i.e: exaspim) in all lowercase and should only contain volumetric image data.

Tile naming convention should be:
tile_X_<xxxx>_Y_<yyyy>_Z_<zzzz>_CH_<wavelength>.ims where xxxx, yyyy, and zzzz are the tile indices of the stack encoded as four digit values.

Add functions/Wrappers to Sample_pose

Need to add set_speed, set_axis_control_mode, and set_ttl_pin_modes. Would be great to have wrappers around this

schema logger

We need a way to record metadata-relevant events. One of the easiest ways of doing this is to record metadata-relevant messages to a separate logger.

location requests should be remapped and exposed in SamplePose level

Functions to implement that return values in sample pose space:

• get_lower_travel_limit
• get_upper_travel_limit
• get_position

joystick axes should be remapped

We should remap the joystick inputs to be something intuitive.

It makes sense to do it somewhere in the tiger_components.py module such that the movement is similar across spim instruments.

Small changes to pre-check functions

Change the input drive to path and write the test file to the local_storage_dir and ext_storage_dir. Also can rename the filename from test.txt to just test.

spim-core/spim_core/spim_base.py

Lines 141 to 192 in b8d9529

	def check_read_write_speeds(self, drive: Path, size='16Gb', bs='1M', direct=1, numjobs=1, ioengine= 'windowsaio',
	iodepth=1, runtime=0):
	"""Check local read/write speeds to make sure it can keep up with acquisition
	:param drive: Drive testing read/write speeds. Usually the local or external storage of instrument
	:param size: Size of test file
	:param bs: Block size in bytes used for I/O units
	:param direct: Specifying buffered (0) or unbuffered (1) operation
	:param numjobs: Number of clones of this job. Each clone of job is spawned as an independent thread or process
	:param ioengine: Defines how the job issues I/O to the file
	:param iodepth: Number of I/O units to keep in flight against the file.
	:param runtime: Limit runtime. The test will run until it completes the configured I/O workload or until it has
	run for this specified amount of time, whichever occurs first
	"""
	test_filename = fr"{drive}\test.txt"
	f = open(test_filename, 'a') # Create empty file to check reading/writing speed
	f.close()
	try:
	speed_MB_s = {}
	for check in ['read', 'write']:
	output = subprocess.check_output(
	fr'fio --name=test --filename={test_filename} --size={size} --rw={check} --bs={bs} '
	fr'--direct={direct} --numjobs={numjobs} --ioengine={ioengine} --iodepth={iodepth} '
	fr'--runtime={runtime} --startdelay=0 --thread --group_reporting', shell=True)
	out = str(output)
	# Converting MiB to MB = (10**6/2**20)
	speed_MB_s[check] = round(
	float(out[out.find('BW=') + len('BW='):out.find('MiB/s')]) / (10 ** 6 / 2 ** 20))
	# converting B/s to MB/s
	acq_speed_MB_s = (self.cfg.bytes_per_image * (1 / 1000000)) * (1 / self.cfg.get_period_time())
	# Go through both speeds and specify if one or both are the problem
	read_too_slow = False
	write_too_slow = False
	if speed_MB_s['read'] <= acq_speed_MB_s:
	read_too_slow = True
	self.log.warning(f'{drive} read speed too slow')
	if speed_MB_s['write'] <= acq_speed_MB_s:
	write_too_slow = True
	self.log.warning(f'{drive} write speed too slow')
	if read_too_slow or write_too_slow:
	raise
	except subprocess.CalledProcessError:
	self.log.warning('fios not installed on computer. Cannot verify read/write speed')
	finally:
	# Delete test file
	os.remove(test_filename)

Also change the previous check memory resources function to public.