Coder Social home page Coder Social logo

pinkshnack / temul Goto Github PK

View Code? Open in Web Editor NEW
13.0 2.0 11.0 144.55 MB

Functions for analysis of high resolution electron microscopy and spectroscopy data.

Home Page: https://temul-toolkit.readthedocs.io

License: GNU General Public License v3.0

Python 98.20% MATLAB 1.80%
python science tem stem electron-microscopy polarisation polarization atomic

temul's Introduction

TEMUL

Functions for image and signal processing, based on the packages Scikit-image, Hyperspy, Atomap, PyPrismatic, ASE, periodictable etc. Written by members of the TEMUL group from the University of Limerick, Ireland (though we welcome all help, hints and contributions!)

Documentation & Installation

pip install temul-toolkit

For full installation instructions and documentation, go to temul-toolkit.readthedocs.io.

To use the vast majority of the temul functionality, import it from the api module::

import temul.api as tml

Interactive Notebooks in the Browser

Jupyter Notebooks and MyBinder allow us to analyse data in the browser without needing any downloads using the below "launch binder" button.

Binder

If the button does not work, try this link instead. You also share this link with others for quick access.

  • To run through some code documentation tutorials, go to the "code_tutorials" folder after clicking the above button.

  • To analyse data published in the listed scientific papers, go to the "publication_examples" folder after clicking the above button.

Example Publications that have used the TEMUL Toolkit

Publication Details Folder Location in TEMUL
M. Hadjimichael, Y. Li et al, Metal-ferroelectric supercrystals with periodically curved metallic layers, Nature Materials 2021 publication_examples/PTO_supercrystal_hadjimichael
K. Moore et al Highly charged 180 degree head-to-head domain walls in lead titanate, Nature Communications Physics 2020 publication_examples/PTO_Junction_moore
K. Moore et al Charged Domain Wall and Polar Vortex Topologies in a Room-Temperature Magnetoelectric Multiferroic Thin Film, ACS Appl. Mater. Interfaces 2022, 14, 4 Coming soon
E.N. O'Connell et al TopoTEM: A Python Package for Quantifying and Visualizing Scanning Transmission Electron Microscopy Data of Polar Topologies, Microsc Microanal. 2022 doi:10.1017/S1431927622000435 URL Arxiv Version (to be updated while M&M version is not open access (it will be soon!) See the Docs

Cite TEMUL Toolkit/TopoTEM

To cite the latest TEMUL toolkit, use the following DOI:

  • DOI

To cite all TEMUL Toolkit versions, use this DOI:

  • DOI

You can also cite the TopoTEM publication if you only worked with polarisation-type analysis.

  • See the TopopTEM publication here. It is annoyingly still not open access. Until that point, see the ArXiv link about in the table, or contact me via the Discussion tab above.

Information for developers

Linting (PEP8)

We use flake8 to enforce coding style:

pip install -r tests/requirements.txt
flake8 temul
flake8 tests
flake8 docs

Testing

Tests are carried out with pytest:

pip install -r tests/requirements.txt
pytest tests

Documentation (docs)

To build the docs, do the following::

pip install -r docs/requirements.txt
sphinx-build docs _build  # open "index.html" in the "_build" directory

Make sure that the docs build locally and online (if you have the rights to check that).

Incrementing version

temul gets its version from the latest git tag. If you think that a new version should be published, create a tag on the master branch (if you have the necessary permissions to do so):

git tag -a 'x.y.z' -m 'new tag'
git push --tags <remote>

GitHub Actions will then automatically build source package and wheels and publish them on PyPI.

temul's People

Contributors

eoinscope avatar michaelhennessyjr avatar pinkshnack avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

temul's Issues

Expand README with install from sources.

The README could be made better and clearer:

  • How to install the package from sources with difference branches/tags
  • How to release a version, but the whole story i.e. make sure lint + docs + tests pass. Confirm that the tar file isn't >100MB locally. Create a release on Github (not really necessary but why not!).

See #61

Update `simulation` module with latest simulation software versions.

Problem

-The current simulation module isn't very modular in that it relies on certain external packages and isn't easily extensible for new packages (e.g. ABTEM), which users might like to use. Therefore its functions need to be broken up and a generalised 'simulate' function should be implemented with the software to be used as a parameter e.g. def simulate(software='prismatic', algorithm='multislice' ... .
-The current TEMUL Toolkit functionality for dealing with simulations is limited to pyprismatic version 0.1.1 (an old one).
It has updated its file format use and we should keep up.

What needs to be done

  • Break up simulation functions so that there are general ones for running simulations and specific ones that handle each software (e.g. pyprismatic, abtem).
  • Warp these general functions as methods in the Model Refiner class for ease-of-use.
  • Add simulation parameters as a dictionary attribute in the Model Refiner class. Then, the user just has to set all parameters in the model refiner as a dict and is free to simulate with whatever software they wish. By parameters here I mean simulation parameters, not python parameters.

Tests are needed

Problem

We don't have any unit tests.

What needs to be done

We have many docstring example tests, so these can be used as a basis for designing unit tests. But more than this are needed, especially for the complicated functions in model_creation.

Best to have a PR for adding tests to each module.

Implement Gaussian Mixture Model for assignment of atom configurations

As discussed with @PinkShnack, it would be better to have a model fitting procedure for initial assignment of atom configurations, rather than the current "hard intensity limit" method. This would help measure error in atom assignments.

It should calculate the intensity of each atom site in a sublattice, plot a histogram of the intensities, and then fit one gaussian to the histogram for each type of atom configuration. Gaussian overlaps are ambiguous - could take some user input to decide how to interpret them.

Might not work for particularly noisy images, as individual histogram peaks might be difficult to detect.

What needs to be done:

  • Create a function to plot histogram of sublattice intensities, fit gaussians based on element lists, and assign elements to each site.
  • Add function to Model Creation module.
  • Wrap function as a method into the Model Refiner class.

Refactor `plot_polarisation_vectors` into a Class.

Overview

The plot_polarisation_vectors function is quite useful for plotting data, but users are limited by the fact that it is a single function and has limited parameters right now. **kwargs can help with this, but it would just make each called function more confusing. It is not well designed and not extensible!

After #70 and some other examples, this has become even more apparent. Users want to be able to manipulate the plots. Some of the issue lies with matplotlib's plotting system, but most of the problems are due to the plot_polarisation_vectors design.

Solution

I will likely try to refactor plot_polarisation_vectors into a class that is more flexible and extensible, and not limited to parameters.

  • For example, it can simply have a norm attribute so that the user can set any matplotlib.colors.colors.Normalize class, which will then be used for any plotting. This is much more user friendly, as one can then play with the underlying functionality freely, rather than relying on someone having the time to add parameters to a function.
  • Most of the underlying functionality can be done this way, with clearly stated defaults.
  • It also makes things easily to test, as each attribute can be individually called and checked.

To do:

  • Deprecation warnings for plot_polarisation_vectors.
  • A new class, perhaps named PolarisationPlotter, which includes setters for setting the underlying functionality of the plotting parameters. See the norm example above.
  • Docstrings for each method
  • Tests for all methods
  • Docs examples showing extensive usecases
  • New description in README.

Acceptance Criteria

  • Python class that replaces and improves all functionality of plot_polarisation_vectors.
  • All class methods tested
  • Docs examples for users

Reorder toolkit code into TopoTEM

TopoTEM should include

  • lattice_structure_tools
  • polarisation
  • some signal_plotting and signal_processing tools

The code base and examples should still work. ideally, only ever import temul with tml

polarisation Colormap legend

the colormaps are great for displaying polarisation but when you add the legend, it onlt has the colours with the angles lablelled.
Could you add arrows in the colourbar beside the labels to make it more intuitive pleaseee

Add default assignment to sort_sublattice_intensities

If sort_sublattice_intensities fails to assign an elemental configuration to a given site, it either assigns the highest intensity configuration or the most common sublattice element (found using get_most_common_sublattice_element). This works well, except for when the most common sublattice element is blank (''), as the function will find all the blanks and replace them with blanks.

It shouldn't be the case that the most common sublattice element is blank, but it might be useful to include a parameter in the function for a standard element to use as a default assignment for cases like this.

TEMUL simulations through PyPrismatic

Hi @PinkShnack

PyPrismatic has undergone an update from 1.1.16 to 0.2.1
The update has disturbed some of the TEMUL functionality, through name changes of functions etc

simulations.py needs some updating

Thnks,
Eileen

Combine `model_creation` into the `Model Refiner` class

Problem

After talking with @michaelhennessyjr (or rather, after trying to show Mike how the model creation works), I have decided to put as much of the model creation into the Model Refiner class.

This will make creation and refinement of models easier. And it will make folding the necessary 2D material model creation workflow into Atomap.

What would need to be done:

  • Break up model creation functions and place them into the model refiner.
    • Make sure to add a note linking the old and new functions to each other.
    • Add a workflow example(s) to the documentation and make a jupyter notebook.
  • Add all the scaling factors as a dict attribute in the Model Refiner.
  • Update the model file outputs to the latest. (probably using ASE and Atomap sublattice).
  • Use the latest version of Atomap for assignment of elements and z_height (dict format, rather than current one in Atomap dev 0.1.2 version). Should be its own issue.

Docs failing due to mentioned of `temul.polarisation`

The readthedocs autobuild is trying to install temul.polarisation which no longer exists. My fault because I merged before checking the new docs!

See also:

WARNING: autodoc: failed to import class 'model_refiner.Model_Refiner' from module 'temul'; the following exception was raised:
No module named 'pyprismatic'
autodoc: failed to import module 'polarisation' from module 'temul'; the following exception was raised:
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/temul-toolkit/envs/latest/lib/python3.7/site-packages/sphinx/ext/autodoc/importer.py", line 154, in import_module
    __import__(modname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/temul-toolkit/checkouts/latest/temul/polarisation.py", line 3, in <module>
    "\n WARNING: "
DeprecationWarning: 
 WARNING: To use the polarisation functionality please use `import temul.api as tml`. The temul.polarisation module is deprecated and is now `temul.topotem.polarisation.


WARNING: autodoc: failed to import module 'simulations' from module 'temul'; the following exception was raised:
No module named 'pyprismatic'
/home/docs/checkouts/readthedocs.org/user_builds/temul-toolkit/checkouts/latest/temul/signal_processing.py:docstring of temul.signal_processing.return_fitting_of_1D_gaussian:6: WARNING: Unexpected section title.

Add simulation package(s) as abstract class input to model refiner

We would like to revamp the model refiner class. IT shouldn't rely on one simulation software. We could do that like so:

  1. Create an abstract simulator class that defines what functionality is required for use with the ModelRefiner.
    • Take a specific image (maybe numpy array!)
    • Take specific image attr dictionary
    • Take specific simulation parameters as dict
  2. Create classes for simulation software, such as abtem and pyprismatic.
    • How could these be implemented without needing these (or more) simulation packages to be dependencies?
      • Could be possible to have these as an "extra" dependency, or just as example classes on the docs?
    • If the simulation classes are not in our codebase we can't test them...

Add simulate_with_abtem function

Abtem should be added to toolkit - I've got simulate_with_abtem and convert_abtem_simulation_to_hspy_signal which could be added to Simulations module, and create_simulation_abtem as a method in the MR class. Should probably just have one generic "simulate" function that takes the software as a parameter and simulates using the relevant function.

Issue about the 'colormap' plot type

**Hello,

I am trying to learn this tools but I face with some problems about the colormap plot.

The issue occured with I run this code:**

tml.plot_polarisation_vectors(x, y, u, v, image=image,
unit_vector=False, save=None,
plot_style='colormap',
overlay=False, title='Vector Arrows', cmap='viridis',
monitor_dpi=50)

It gives me an error as following:


ValueError Traceback (most recent call last)
Cell In[7], line 4
1 #Vector magnitude plot with colormap viridis:
----> 4 tml.plot_polarisation_vectors(x, y, u, v, image=image,
5 unit_vector=False, save=None,
6 plot_style='colormap', cbar_vmax=1,cbar_vmin=0.1,
7 overlay=False, title='Vector Arrows', cmap='viridis',
8 monitor_dpi=50)

File D:\Python\Python39\lib\site-packages\temul\topotem\polarisation.py:584, in plot_polarisation_vectors(x, y, u, v, image, sampling, units, plot_style, overlay, unit_vector, vector_rep, degrees, angle_offset, save, title, color, cmap, cbar_vmin, cbar_vmax, alpha, image_cmap, monitor_dpi, no_axis_info, invert_y_axis, ticks, scalebar, antialiased, levels, remove_vectors, quiver_units, pivot, angles, scale_units, scale, headwidth, headlength, headaxislength, width, minshaft, minlength)
582 sm = plt.cm.ScalarMappable(cmap=cmap, norm=norm)
583 sm.set_array([])
--> 584 cbar = plt.colorbar(mappable=sm, fraction=0.046, pad=0.04,
585 drawedges=False)
586 cbar.set_ticks(ticks)
587 cbar.ax.set_ylabel(vector_label)

File ~\AppData\Roaming\Python\Python39\site-packages\matplotlib\pyplot.py:2326, in colorbar(mappable, cax, ax, **kwargs)
2321 if mappable is None:
2322 raise RuntimeError('No mappable was found to use for colorbar '
2323 'creation. First define a mappable such as '
2324 'an image (with imshow) or a contour set ('
2325 'with contourf).')
-> 2326 ret = gcf().colorbar(mappable, cax=cax, ax=ax, **kwargs)
2327 return ret

File ~\AppData\Roaming\Python\Python39\site-packages\matplotlib\figure.py:1285, in FigureBase.colorbar(self, mappable, cax, ax, use_gridspec, **kwargs)
1283 if cax is None:
1284 if ax is None:
-> 1285 raise ValueError(
1286 'Unable to determine Axes to steal space for Colorbar. '
1287 'Either provide the cax argument to use as the Axes for '
1288 'the Colorbar, provide the ax argument to steal space '
1289 'from it, or add mappable to an Axes.')
1290 fig = ( # Figure of first axes; logic copied from make_axes.
1291 [*ax.flat] if isinstance(ax, np.ndarray)
1292 else [*ax] if np.iterable(ax)
1293 else [ax])[0].figure
1294 current_ax = fig.gca()

ValueError: Unable to determine Axes to steal space for Colorbar. Either provide the cax argument to use as the Axes for the Colorbar, provide the ax argument to steal space from it, or add mappable to an Axes.

I thinks the problem is about the mappable. It turns out that the code thinks the mappable is none. I try to solve this problem but I can't. I am not sure why this issue occur. Is it because of the python or matplotlib version? (My python is 3.9 and matplotlib is 3.8)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.