Coder Social home page Coder Social logo

das-developers / das2py Goto Github PK

View Code? Open in Web Editor NEW
6.0 4.0 4.0 3.66 MB

An efficient space physics data client for python

Home Page: https://das2.org/das2py

License: MIT License

Makefile 2.28% Python 82.34% Batchfile 1.14% HTML 0.21% C 13.85% Shell 0.18%
das2 python magnetosphere space-physics

das2py's Introduction

das2py

Das2 servers typically provide data relevant to space plasma and magnetospheric physics research. To retrieve data, an HTTP GET request is posted to a das2 server by a client program and a self-describing stream of data values covering the requested time range, at the requested time resolution, is provided in the response body. This package, das2py provides an efficient space physics data client for python. Streams are parsed and stored as NumPy arrays using a C extension, avoiding data copies and conversions.

Anaconda Package

Anaconda Package

Pre-build versions of das2py are available from Anaconda. If you're working in an Anoconda or Miniconda python 3 environment these are easier to install as no C compiler is required. To install the conda package run the command:

(base) $ conda install -c dasdevelopers das2py

The anaconda package automatically pulls in das2C, pycdf, and pthreads4w as needed.

If this works then test using:

(base) $ wget https://raw.githubusercontent.com/das-developers/das2py/master/examples/ex05_mex_marsis_query_by_angle.py
(base) $ python ex02_galileo_pws_spectra.py

If this command produces a plot similar to the following...

...then das2py is installed, and you can skip building the software and head straight the example program below.

Prerequisite

Compilation and installation of das2py has been tested on Linux, Windows, MacOS using both Python 2 and Python 3. The following packages are required to build das2py:

  • Das2C - Version 2.3 or above. Need not be installed, but must be built
  • NumPy - Version 1.10.1 or above
  • MatplotLib++ - For plotting data (optional, recommended)
  • CDF - For writing CDF files (optional)

Pre-requisite package install commands are give below.

$ sudo apt install python3-setuptools python3-dev python3-numpy # debian

New build instructions (wheel)

In this version, almost no environment variables are needed Below, das2C is built adjacent to das2py so that it can be included in das2py.

# (Works on Linux, more steps needed on Windows & MacOS. 
#  See buildfiles/pypi/ReadMe.md for details)

# First build and test das2C, installation is not necessary
git clone [email protected]:das-developers/das2C.git
cd das2C
env N_ARCH=/ make 
env N_ARCH=/ make test
cd ../

# Now build das2py using the PIP of your choice in an adjacent directory
git clone [email protected]:das-developers/das2py.git
cd das2py
env DAS2C_LIBDIR=$PWD/../das2C/build. DAS2C_INCDIR=$PWD/../das2C python3.9 -m build

That's it! Now you have a wheel file that can be installed where ever you like. The included setup.py instructs the python setuptools module to staticlly link das2C. So the wheel is self contained.

To install your new wheel into the user-local area:

pip3.9 install ./dist/das2py-2.3.0-cp310-cp310-linux_x86_64.whl
ls .local/bin/das_verify
ls .local/lib/python3.9/site-packages/das2
ls .local/lib/python3.9/site-packages/_das2*

To test it by validating one of the example files...

das_verify das2py/test/ex96_yscan_multispec.d2t

and to generate a plot of Cassini electron cyclotron frequencies.

python3.9 das2py/examples/ex09_cassini_fce_ephem_ticks.py 2017-09-14
okular cas_mag_fce_2017-09-14.png # Or whatever PNG viewer you like

Old Build and Install

The old makefile based build and install wrapper is still supported. It's based on distutils, and it works for Python 2.7 and up. For newer versions of python the pip-wheel build is recommended, for Python2 support read on...

Decide where you want to install das2py. In the example below I've selected /usr/local/lib/python3.9/site-packages but any location is fine so long as it is on the PYTHONPATH or you are willing to add it to your PYTHONPATH.

# Where to find the das2C static library
$ export DAS2C_INCDIR=$HOME/git/das2C
$ export DAS2C_LIBDIR=$HOME/git/das2C/build.  #last dot is not a typo

# Which python version to use
$ export PYVER=3.9

# Where you want to install the files
$ export INST_HOST_LIB=/usr/local/lib/python3.9
$ export INST_EXT_LIB=/usr/local/lib/python3.9

# Build and test
$ make           # <-- If only using system packages
$ make local     # <-- If using numpy or others from $HOME/.local
                 #     Do not use for RPM/DEB builds
$ make test

# Check install location, then install
$ make -n install
$ make install

First program

The following small program demonstrates how to query for data and generate a plot using das2py.

Query a URI for reduced resolution data

import das2
src = das2.get_source( 'tag:das2.org,2012:site:/uiowa/galileo/pws/survey_electric/das2' )
dataset = src.get( {'time' : ('1997-05-07T15:00', '1997-05-07T17:00', 4.0)} )[0] 
  • Servers come and go. The federated catalog provides stability for application code by maping URIs to data sources.
  • Browse for URIs using a das2 catalog browser, or using das2py to get the root node (see example 11).

Access quantities by physical dimensions

print(dataset)

vX = dataset['time']['center']
vY = dataset['frequency']['center']
vZ = dataset['electric']['center'] 
  • Datasets contain dimensions. Dimensions contain variables. Each variable in a dimension serves a purpose, the most common is to define the center point of a coordinate or measurement.
  • Array dimensions are not confused with physical dimensions. Dataset meaning is not tied to any particular array morphology.

Use Matplotlib to generate an image.

import matplotlib.pyplot as pyplot
import matplotlib.colors as colors

fig, ax = pyplot.subplots()
scaleZ = colors.LogNorm(vmin=vZ.array.min(), vmax=vZ.array.max())
ax.pcolormesh(vX.array, vY.array, vZ.array, norm=scaleZ, cmap='jet')
pyplot.show() 

Reporting bugs

Please use the github.com issue tracker report any problems with the library. If you've fixed a bug, 1) thanks!, 2) please send a pull request so that your updates can be merged into the main project.

das2py's People

Contributors

baptistececconi avatar cpiker avatar kaiwens7 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

das2py's Issues

PIP Package

Currently the only pre-packaged version of das2py with it's das2C dependency included is an Anaconda3 package. Producing a PIP package, presumably in wheel format, is a high priority.

Add Waves/burst data

Add functions that read and plot Juno/Waves/burst data from PDS (assigned to CorentinLouis)

Examples not complete

The following example scripts are not finished:

  • ex06_astro_lwa1_multiscale.py
  • ex07_cassini_rpws_query_opts.py
  • ex09_van_allen_probes_mag_ephem.py

These examples need to be completed and a sample plot should be provided.

SSL Error with das2py

I'm having a problem with das2py it seems that I cannot reach any data on the server, when I try to reach any node (except iowa) I get the following message

src = das2.get_node('tag:das2.org,2012:site:/uiowa/juno')INFO: error:1408F10B:SSL routines:ssl3_get_record:wrong version number

INFO: error:1408F10B:SSL routines:ssl3_get_record:wrong version number

Traceback (most recent call last):
File "", line 1, in
File "/opt/anaconda3/envs/python37/lib/python3.7/das2/init.py", line 250, in get_node
dDef = _das2.get_node(sPathId)
_das2.Error: Node U. Iowa (URI 'tag:das2.org,2012:site:/uiowa') has no child node that starts with juno
(Reported from das2/node.c, line 539)

Surprisingly, src = das2.get_node('tag:das2.org,2012:site:/uiowa') doesnt return an error and in that case src['juno'].props returns

{'name': 'Juno',
'title': 'Jupiter polar orbiter - Launch date 2011-08-05',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/juno.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/juno'}

If that can help I have this :

src.props

{'catalog': {'cassini': {'name': 'Cassini',
'title': 'Saturn System Explorer - Orbit Insertion 2004-06-30',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/cassini.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/cassini'},
'de': {'name': 'DE',
'title': 'Earth polar orbiters - Launch date 1981-08-03',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/de.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/de'},
'earth': {'name': 'Earth',
'title': 'Earth based observatories',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/earth.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/earth'},
'galileo': {'name': 'Galileo',
'title': 'Jupiter equatorial orbiter - Launch date 1989-10-13',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/galileo.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/galileo'},
'isee': {'name': 'ISEE',
'title': 'International Sun-Earth Explorers - Launch date 1977-10-22',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/isee.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/isee'},
'juno': {'name': 'Juno',
'title': 'Jupiter polar orbiter - Launch date 2011-08-05',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/juno.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/juno'},
'mars_express': {'name': 'Mars Express',
'title': 'Mars polar orbiter - Launch date 2003-06-02',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/mars_express.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/mars_express'},
'maven': {'name': 'Maven',
'title': 'Mars Atmosphere and Volatile Evolution Mission - Orbit Insertion 2014-09-22',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/maven.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/maven'},
'stereo': {'name': 'Stereo',
'title': 'Dual Solar Terrestrial Relation Observatories - Launch date 2006-10-26',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/stereo.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/stereo'},
'van_allen_probes': {'name': 'Van Allen Probes',
'title': "Twin probes of the Earth's inner and outer radiation belts, launched on 2012-08-30",
'type': 'Catalog',
'urls': ['https://emfisis.physics.uiowa.edu/~cwp/van_allen_probes.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/van_allen_probes'},
'voyager': {'name': 'Voyager',
'title': 'Dual Spacecraft Mission to the Outer Planets, and Beyond',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/voyager.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/voyager'},
'wind': {'name': 'Wind',
'title': 'WIND Solar-Terrestrial Mission - Launch date 1994-11-01',
'type': 'Catalog',
'urls': ['http://das2.org/catalog/das/site/uiowa/wind.json'],
'_path': 'tag:das2.org,2012:site:/uiowa/wind'}},
'name': 'U. Iowa',
'title': 'University of Iowa, Department of Physics and Astronomy',
'type': 'Catalog',
'version': '0.4',
'_url': 'https://raw.githubusercontent.com/das-developers/das-cat/master/cat/das/site/uiowa.json',
'_path': 'tag:das2.org,2012:site:/uiowa'}

I try to update das2 numpy and spacepy but thats didn't help.

Maybe it is related to the fact that when I want to edit the url of a plot (from Juno data) on a saved vap file on autoplot I have this
image

However, I managed to fix this by changing the adress with https instead of http.

das2.convert for undefined units error

The following code snippet fails with a hard segfault:

import das2
das2.convert(1.0, "us2000", "TT2000")

This is was not listed as a das2C issue because it is most likely triggered by improper module initialization.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.