neptuneprojects / risworkflow

This repository contains data and instructions for how to implement the RISCluster package.

RISWorkflow

This repository contains data and instructions for how to implement deep embedded clustering (DEC) and Gaussian mixture model (GMM) clustering of seismic data recorded on the Ross Ice Shelf, Antarctica from 2014-2017. This package is an accompaniment to the paper published in the Journal of Geophysical Research: Solid Earth.

The workflow is as follows:

1. Load and pre-process seismic data.
2. Use a convolutional auto-encoder to reduce dimensionality of input data.
3. Perform GMM clustering on latent data.
4. Perform DEC clustering on latent data.
5. Compare results in latent space and data space.

---

Installation

1. Install RISCluster using instructions contained in the RISCluster repository readme.md.
2. Clone this repository to your desired working directory.
3. Download environmental data and unzip contents into /Data.

---

Usage

The Jupyter notebook Workflow.ipynb contains an end-to-end workflow control that guides the user through all steps of the project, including downloading and pre-processing the seismic data. Required directories, configuration files, and command-line scripts are generated within the notebook. For main routine execution, commands are copied from the Jupyter notebook into a terminal window.

Downloading and processing seismic data can take a long time. For access to the pre-processed seismic data set (16 GB), please contact me and we can arrange how best to transfer the file.

---

References

William F. Jenkins II, Peter Gerstoft, Michael J. Bianco, Peter D. Bromirski; Unsupervised Deep Clustering of Seismic Data: Monitoring the Ross Ice Shelf, Antarctica. Journal of Geophysical Research: Solid Earth, 30 August 2021; doi: https://doi.org/10.1029/2021JB021716

Dylan Snover, Christopher W. Johnson, Michael J. Bianco, Peter Gerstoft; Deep Clustering to Identify Sources of Urban Seismic Noise in Long Beach, California. Seismological Research Letters 2020; doi: https://doi.org/10.1785/0220200164

Junyuan Xie, Ross Girshick, Ali Farhadi; Unsupervised Deep Embedding for Clustering Analysis. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, 2016; https://arxiv.org/abs/1511.06335v2

---

Author

Project assembled by William Jenkins
wjenkins [@] ucsd [dot] edu
Scripps Institution of Oceanography
University of California San Diego
La Jolla, California, USA

risworkflow's Issues

Workflow.ipynb import obspy crashes

Hi!
Trying to run RISCluster_CPU on Linux.
After successfully creating the environment ipython notebook crashes due to problem with obspy importing when importing one of RISCluster packages (import obspy crashes). Seems like obspy vs numpy versions conflict:
'numpy.int64' object has no attribute 'split'

Currently in reproduced env:
obspy 1.2.2 py38h6c62de6_2 conda-forge
numpy 1.22.1 py38h6ae9a64_0 conda-forge

I also tried to create a modified environment changing blindely obspy to obspy==1.1.1 in the yml file however that causes a massive amount of conflicts and environment can't be solved.

I think you need to cross-check verson of packages in your environment and update yml file accordingly.
Looking forward to run RISCluster!

Preprocess step fails - 0 files saved

Hi!

Still trying to run the code...
After successful data download preprocessing step fails with no error raised:

(RISCluster_CPU) [wgajek@opal RISWorkflow]$ process ./Config/config_preprocess.ini

Processing data. Workers: 4
| | 0.00/0.00 [00:00<?, ?it/s]
Processing complete.
0 files saved to /home/wgajek/RISWorkflow/raw/Preprocessed
0:00:00 elapsed.

The data is downloaded (
'start': '20141201T0200',
'stop': '20141201T0500',) just for trying
The name format is matching actual filenames. The output /Preprocessed/params_preprocess.json is created and looks ok (below).
It seems the code cannot recognize the downloaded data (the paths match - in process.ini the seismo data path is the same as in download scipt).

Preprocessed/params_preprocess.json :
{"mode": "preprocess", "sourcepath": "/home/wgajek/RISWorkflow/raw", "writepath": "/home/wgajek/RISWorkflow/raw/Preprocessed", "catalogue": ".", "parampath": "/home/wgajek/RISWorkflow/raw/Preprocessed", "name_format": "2", "network": "XH", "station": "*", "channel": "HHZ", "taper": "60.0", "prefeed": "60.0", "fs2": "50.0", "cutoff": "[3.0, 20.0]", "T_seg": "None", "NFFT": "None", "tpersnap": "None", "overlap": "None", "output": "acc", "prefilt": "(0.004, 0.01, 500.0, 1000.0)", "waterlevel": "14.0", "detector": "None", "STA": "None", "LTA": "None", "on": "None", "off": "None", "det_window": "None", "num_workers": "4", "verbose": "False", "start": "2014-12-01 02:00:00", "stop": "2014-12-01 05:00:00", "buffer_front": "120.0", "buffer_back": "60.0", "start_processing": "2014-12-01 01:58:00", "stop_processing": "2014-12-01 05:01:00"}