Welcome to EKPy

A repository of analysis and computer control code for various experiments. Image above is an example of data collected and analyzed using this package.

See here for introdution videos

Overview
Installation
Development
Important Links
Change Log
Support
Cite this code
Cited by

Overview

EKPy (formerly ekpmeasure) is a set of control and analysis code designed to help streamline experiments. The basic idea is that in experimental work we often take data from many different sources, store it in different places, have varying degrees meta data associated with the data (even for a single type of data) and somehow(!) we are supposed to make sense of it all. We like to compare across trials, days, experimental conditions, etc. and it is very difficult to keep track of what data is where, and quickly access it when we need it. Often I find that folks end up copying and pasting raw data between excel spreadsheets and if you're not careful you will quickly lose track of which data came from where. This package's goal is to make this all easier.

You may not find the experimental control code as helpful as it is relatively specific to my research in condensed matter physics (though electrical engineers or similar may find it very useful) but the analysis code is for everyone.

At the heart of the analysis is the Dataset class which is a means of manipulating meta data alone in order to locate which actual data you want to analyze. Datasets don't care about what the real data looks like, and they keep track of where different data is stored so it is easy to select which data you want to look at - only then do you retrieve the data. The real data is returned in a Data class which allows you to group by parameters, perform calculations and much more.

I am always improving this repository and if you have suggestions, I appreciate any feedback and or issues (https://github.com/eparsonnet93/ekpmeasure/issues)

See examples for more.

Installation:

If you plan to use EKPy to control your experiments. You will need to install NI-VISA first.

Find the latest build here.

pip install ekpy

or to upgrade to the latest version

pip install -U ekpy

You can also access .whl or .tar.gz files in the dist/ directory directly for installation.

For installation issues, please see Issue Tracker

There are also specific experiments that can be installed following installation of ekpy. Please see the accompanying readme (e.g., here) for such cases.

Development

We welcome new contributors of all experience levels. Please reach out directly ([email protected]) to inqure about getting involved.

Important Links

Official source code repo: https://github.com/eparsonnet93/ekpmeasure
Issue tracker: https://github.com/eparsonnet93/ekpmeasure/issues

Change log

Version 0.1.14

With help form Geo!

Updates to utils to make to improve the function get_number_and_suffix + other updates to include additional instruments and fixes.

Version 0.1.13

Can now group Data. data.groupby
Can now remove nans from data. data.dropna
Improvements to file management in control, saving, and more appropriate trial incrementing in control.experiment
Functions used in data.apply should no longer require the use of iterable data arrays. Please raise an issue if one occurs.

Version 0.1.12

Added functionality to analysis.radiant
Speed increase for data.get_data()
Minor bugs and other fixes

Version 0.1.11

experiment.print_run_function_args is gone. It has been replaced with experiment.show_run_function_help
Added a header of meta data to the default save function for control.experiment and started the process of doing away with meta_data .pkl files in favor of .csv (why did I ever think that was a good idea?)

Version 0.1.10

minor bug fixes and documentation updates

Version 0.1.9

minor bug fixes

Version 0.1.8

2/18/22

bug fixes from 0.1.7 and changed install name to ekpy.

Version 0.1.6

1/23/22

added Data.to_DataFrame() which allows one to convert Data to pandas.DataFrame. Each index of Data will be a single row in the resulting DataFrame.

Version 0.1.5

1/12/22

merge_Datas was replaced. The older version is now concat_Datas as it was really just concatenation, not merging. Merging now has real meaning, to merge a set of similar data objects on a specified definition key. Please report errors as they arise.
similary merge_Datasets is deprecated. One must use concat_Datasets.

Version 0.1.4

12/2/21

Added experiments module with ferroelectric experiments. This contains both relaxation (_relaxation) and switching (_switching) experiments. These consist of self contained jupyter notebooks that can be installed by python -m ekpy.experiments.ferroelectric.<experiment_name>. For more see the experiment specific README for relaxation or switching
Bug fix on Data.sort

11/16/21

Data.apply now allows for dropping data. This can be executed by having the function in .apply return 'None'.
data_array_builder.build now allows one to fix lengths on 1D data by appending nans to make all data arrays the same length.
analysis.plotting.add_legend_element now allows kwarg fontsize

11/19/21

Data can now be saved. Use data.to_ekpdat.
- Can be loaded as analysis.read_ekpdat

Version 0.1.3

11/11/21

Minor fixes for deprecations in control.misc

Version 0.1.2

11/7/21

Speed improvements to Dataset.remove_nonexistent_files
Data objects can now be sorted by a definition parameter

Version 0.1.1

11/2/21

Updates to plotting during experimental control. Now one can simply override the control.experiment method _plot to define how plotting will take place. Here is a brief example of such an override:

from ekpy.control import plotting
from ekpy.control import experiment
import matplotlib.pyplot as plt

class exp(experiment):

	...

	def _plot(self, data, scan_params):
		if hasattr(self, 'fig') and hasattr(self, 'ax'):
			pass
		else:
			fig, ax = plt.subplots()
			self.fig = fig
			self.ax = ax
			
		self.ax.scatter(scan_params['frequency'], np.mean(data['R']), color = 'blue')
		plt.show(self.fig)
		plotting.update_plot(self.fig)

	...

Version 0.1.0

10/24/21

Dataset class is no longer subclass of pandas.DataFrame. This is to limit usage of unsupported functions.
Data class updates including sorting and collapsing.
One can access data or definition directly as an attribute now Data.definition, for example. One can also access pieces of information such as the real data corresponding to p1 as Data.p1 or definition keys, e.g. high_voltage as Data.high_voltage
Experiment class now saves a backup .csv meta data file in addition to the pickle file in order to help with errors related to different pandas versions on various machines.

Support

Code related issues (e.g. bugs, feature requests) can be created in the issue tracker

Maintainer: Eric Parsonnet

Citation

Please cite this work following the CITATION.cff (see here for more details on how to cite.)

Cited By

E. Parsonnet et. al. NonVolatile Electric Field Control of Thermal Magnons in the Absence of an Applied Magnetic Field, Phys. Rev. Lett 129, 087601 (2022).

issue with passing arguments to function in data.apply()

I would have expected this to throw the normal error, where it prints the problem but continues with the apply function on remaining data. Instead it errors out entirely.

data = data.apply(offset_by_delay, pass_defn = True, offseter = .001)

Error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~\Documents\ekpmeasure\src\ekpmeasure\analysis\core.py in apply(self, function_on_data, pass_defn, kwargs_for_function, **kwargs)
    734 					internal_out = {
--> 735                                                 'definition':tmp_out[key]['definition'],
    736                                                 'data':data_function(tmp_out[key]['data'].copy(), defn = tmp_out[key]['definition'], **to_pass)

KeyError: 'offseter'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-175-8f0e65047bf6> in <module>
      1 data = dset_25nm.query('diameter == 23').get_data(groupby = 'delay_ns').mean().apply(fe.reset_time).apply(fe.get_dps)
----> 2 data = data.apply(offset_by_delay, pass_defn = True, offseter = .001)
      3 data.plot(x = 'time', y = 'dp', ax = ax3)

~\Documents\ekpmeasure\src\ekpmeasure\analysis\core.py in apply(self, function_on_data, pass_defn, kwargs_for_function, **kwargs)
    745                         except Exception as e:
    746                                 print('Error in data_function: {} \n{}'.format(data_function.__name__, e))
--> 747                                 print('Skipping data key: {} with defintion: \n{}'.format(key, tmp_out[key]['definition']))
    748 
    749                 return Data(tmp_out)

KeyError: 'offseter'

For reference, the function 'offset_by_delay' is defined here:

def offset_by_delay(data, delay_ns, offseter = .001, **kwargs):
    ida = iterable_data_array(data, 'dp')
    out = data_array_builder()
    
    if len(delay_ns) != 1:
        raise TypeError('more than one delay_ns in this data index!')
        
    delay_ns = list(delay_ns)[0]
    
    for dp in ida:
        indexer = np.log(delay_ns)/np.log(10)
        out.append(dp + indexer*offseter)
        
    data_out = data.copy()
    data_out.update({'dp':out.build()})
    return data_out

eparsonnet93 / ekpmeasure Goto Github PK

ekpmeasure's Introduction