Coder Social home page Coder Social logo

jakewalter / easyquake Goto Github PK

View Code? Open in Web Editor NEW
60.0 6.0 21.0 368.76 MB

Simplified machine-learning driven earthquake detection, location, and analysis.

License: MIT License

Python 100.00%
earthquake-detection obspy tensorflow earthquake seismology

easyquake's Introduction

easyQuake

Simplified machine-learning driven earthquake detection, location, and analysis in one easy-to-implement python package.

For more details, see the documentation: https://easyquake.readthedocs.io/

On most systems you should be able to simply:

pip install easyQuake

To stay on the bleeding edge of updates:

pip install easyQuake --upgrade

Or if you need to tweak something, like the number of GPUs in gpd_predict, you could:

git clone https://github.com/jakewalter/easyQuake.git
cd easyQuake
pip install .

If you find this useful, please cite:

Walter, J. I., P. Ogwari, A. Thiel, F. Ferrer, and I. Woelfel (2021), easyQuake: Putting machine 
learning to work for your regional seismic network or local earthquake study, Seismological Research 
Letters, 92(1): 555โ€“563, https://doi.org/10.1785/0220200226

Requirements

This code leverages machine-learning for earthquake detection with the choice of the GPD (https://github.com/interseismic/generalized-phase-detection), EQTransformer (https://github.com/smousavi05/EQTransformer), or PhaseNet (https://github.com/AI4EPS/PhaseNet) pickers. You should have suitable hardware to run CUDA/Tensorflow, which usually means some sort of GPU. This has been tested on servers with nvidia compute cards and modest multi-core desktop with consumer gaming nvidia card (e.g. Geforce 1050 Ti). The event-mode can be run efficiently enough on a laptop.

  • Most tested configuration includes nvidia-cuda-toolkit, obspy, keras, tensorflow-gpu==2.2, basemap
  • I've found that the the easiest way to install cuda, tensorflow, and keras is through installing Anaconda python and running conda install tensorflow-gpu==2.2
  • Because tensorflow-gpu 2.2 requires python 3.7 (not the latest version), you might find an easier road creating a new environment:
conda create -n easyquake python=3.7 anaconda
conda activate easyquake
conda install tensorflow-gpu==2.2
conda install keras
conda install obspy -c conda-forge
pip install easyQuake

Running easyQuake

The first example is a simple one in "event mode" - try it:

from easyQuake import detection_association_event

detection_association_event(project_folder='/scratch', project_code='ok', maxdist = 300, maxkm=300, local=True, machine=True, latitude=36.7, longitude=-98.4, max_radius=3, approxorigintime='2021-01-27T14:03:46', downloadwaveforms=True)

This next example runs easyQuake for a recent M6.5 earthquake in Idaho for the 2 days around the earthquake (foreshocks and aftershocks). The catalog from running the example is in the examples folder: https://github.com/jakewalter/easyQuake/blob/master/examples/catalog_idaho_2days.xml

If you don't have a suitable computer, try it in Google Colab Open In Colab

from easyQuake import download_mseed
from easyQuake import daterange
from datetime import date
from easyQuake import combine_associated
from easyQuake import detection_continuous
from easyQuake import association_continuous

from easyQuake import magnitude_quakeml
from easyQuake import simple_cat_df

import matplotlib.pyplot as plt
maxkm = 300
maxdist=300
lat_a = 42
lat_b = 47.5
lon_a = -118
lon_b = -111


start_date = date(2020, 3, 31)
end_date = date(2020, 4, 2)

project_code = 'idaho'
project_folder = '/data/id'
for single_date in daterange(start_date, end_date):
    print(single_date.strftime("%Y-%m-%d"))
    dirname = single_date.strftime("%Y%m%d")
    download_mseed(dirname=dirname, project_folder=project_folder, single_date=single_date, minlat=lat_a, maxlat=lat_b, minlon=lon_a, maxlon=lon_b)
    detection_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, single_date=single_date, machine=True,local=True)
    #run it with EQTransformer instead of GPD picker
    #detection_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, machine=True, machine_picker='EQTransformer', local=True, single_date=single_date)
    #PhaseNet
    #detection_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, machine=True, machine_picker='PhaseNet', local=True, single_date=single_date)
    association_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, maxdist=maxdist, maxkm=maxkm, single_date=single_date, local=True)
    ### IMPORTANT - must call the specific picker to create association and catalogs specific to that picker within each dayfolder!!
    #association_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, maxdist=maxdist, maxkm=maxkm, single_date=single_date, local=True, machine_picker='EQTransformer')
    #association_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, maxdist=maxdist, maxkm=maxkm, single_date=single_date, local=True, machine_picker='PhaseNet')

cat, dfs = combine_associated(project_folder=project_folder, project_code=project_code)
#cat, dfs = combine_associated(project_folder=project_folder, project_code=project_code, machine_picker='EQTransformer')
#cat, dfs = combine_associated(project_folder=project_folder, project_code=project_code, machine_picker='PhaseNet')
cat = magnitude_quakeml(cat=cat, project_folder=project_folder,plot_event=True)
cat.write('catalog_idaho.xml',format='QUAKEML')


catdf = simple_cat_df(cat)
plt.figure()
plt.plot(catdf.index,catdf.magnitude,'.')

Tips for successful outputs

Within your systems, consider running driver scripts as nohup background processes nohup python ~/work_dir/okla_daily.py &. In this way, one could cat nohup.out | grep Traceback to understand python errors or grep nohup.out | Killed to understand when the system runs out of memory.

Video intros to easyQuake

Most recent updates, recorded for the 2021 SSA Annual meeting: https://www.youtube.com/watch?v=bjBqPL9pD5w

Recorded for the fall 2020 Virtual SSA Eastern Section meeting: https://www.youtube.com/watch?v=coS2OwTWO3Y

About EasyQuake

Stay up to date on the latest description of EasyQuake contents: https://easyquake.readthedocs.io/en/latest/About.html

Running easyQuake with SLURM

If you have access to shared computing resources that utilize SLURM, you can drive easyQuake by making a bash script to run the example code or any code (thanks to Xiaowei Chen at OU). Save the following to a drive_easyQuake.sh and then run it

#!/bin/bash
#
#SBATCH --partition=gpu_cluster
#SBATCH --ntasks=1
#SBATCH --mem=1024
#SBATCH --output=easyquake_%J_stdout.txt
#SBATCH --error=easyquake_%J_stderr.txt
#SBATCH --time=24:00:00
#SBATCH --job-name=easyquake
#SBATCH [email protected]
#SBATCH --mail-type=ALL
#SBATCH --chdir=/drive/group/user/folder
conda init bash
bash
conda activate easyquake
python idaho_example.py

Version brief notes

Version 1.3 (11/22/2022) = PhaseNet now included, in addition to GPD and EQTransformer pickers. Numerous other bugs squashed.

Version 1.2 (8/1/2022) - Rewrote the non-ML picker to be easier to work with (recursive_sta_lta from obpsy) and include input of those parameters within detection_continuous function.

Version 0.9 (2/23/2022) - Modules to cut easyQuake event waveforms from continuous data (cut_event_waveforms) and module for converting easyQuake catalog (or any QuakeML-formatted catalog) to HDF5 (quakeML_to_hdf5) for training new ML models

Version 0.8 (7/30/2021) - Several major bug fixes and improved controls for Hypoinverse location

Verson 0.6 (2/24/2021) - Implemented choice of GPD or EQTransformer pickers for the picking stage

Version 0.5 (2/10/2021) - includes embedded hypoinverse location functionality, rather than the simple location with the associator.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Acknowledgments

easyquake's People

Contributors

jakewalter avatar kennedydane avatar longmho avatar raymond123ng avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

easyquake's Issues

Question about the available models

Hello,

are the .h5 files relative to the original implementation of each model, i.e. trained with the original dataset and having the original weights of the models? Or did you retrain each of the models?

Q: Integrating custom picker picks with EasyQuake's phase associator

Hello,

I am using a seismic event picker that isn't currently integrated with EasyQuake and I can produce picks in a format similar to EasyQuake's expected input (e.g., "MB NDMT EHE S 2020-04-01T20:56:50.730000").

I'm interested in utilizing EasyQuake's phase associator with my picker's output. Could you provide guidance on how to properly use the phase associator with a custom "stalta_picks.out" file generated by an external picker?

The main problem I see is that my picker doesn't save the channel information, can I just skip channel using an asterisk or so?

Thank you for your support.
D

Request/suggestion regarding cut_event_waveforms

What a great tool! It would be very useful to allow the 'cut_event_waveforms' module to clip the traces for all available waveforms for that day, not just those that triggered and were associated. I am trying to code this up myself but thought I would mention it as it might be useful to others as well.

Using 5 letter stations doesn't work

Hello, first off I want to thank you for developing such a great earthquake detection system.
I do have an issue though.
How can I import 5 letter station codes?
Most of the stations I use have 5 letter station codes, but they are not handled correctly by the program.
For example in the sta file the following happens:
LBTB2500.91S02535.80E1148
RTLG2121.71S02112.94E1198
How ever the second station should be GRTLG.
Any help or suggestions?
Thanks.

Error running the code

Hi authors of easyQuake,

I have encountered the following error when running the code, it seems to have some problem loading the model,
AttributeError: 'str' object has no attribute 'decode'
After searching for the error on google, I ran the following command:
pip install 'h5py==2.10.0' --force-reinstall
Then the program could run normally.

I guess there is something incompatible with the installed version of Keras and h5py, in case someone else have the same problem.

Thank you,
Chenyu Li

Issue Running Examples

Hello Jake,

I am attempting to run easyQuake and I have ran into a few preliminary challenges. One is though I have conda forge as an accessible channel, I cannot seem to download tensorflow-gpu==2.1.

Secondly, I do not have a machine with a GPU. Your write up does not explicitly state that a gpu is necessary. Can easyQuake be ran with a CPU?

Third, I am attempting to work through the idaho_example.py and I receive an error for the project_folder variable in the download_mseed(), detection_continous(), and association_sontinous() functions. When I try to run these this is what is returned: OSError: [Errno 30] Read-only file system: '/data'. Tho pointers "data/id" where pre-populated in the file, is this something I should change to a directory that exists or is this used to create a new directory to populate the downloaded files?

Bryan

if no events detected in time chunk --> error creating gpd_picks.out --> process exits

Hi Jake, I am falling into an error in which, if no events are detected during one time chunk (With EQTransofrmer as machine picker), the gpd_picks.out file is not created and the process exits without passing to the next time-chunk.

Th error I get is the following:


FileNotFoundError Traceback (most recent call last)
in
27 dirname = single_date.strftime("%Y%m%d")
28 download_mseed(dirname=dirname, project_folder=project_folder, single_date=single_date, minlat=lat_a, maxlat=lat_b, minlon=lon_a, maxlon=lon_b)
---> 29 detection_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, single_date=single_date, machine=True, machine_picker= 'EQTransformer', local=True)
30 association_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, maxdist=maxdist, maxkm=maxkm, single_date=single_date, local=True)
31

/usr/local/Caskroom/miniconda/base/envs/easyquake/lib/python3.7/site-packages/easyQuake/easyQuake.py in detection_continuous(dirname, project_folder, project_code, local, machine, machine_picker, single_date, latitude, longitude, max_radius)
530 fullpath2 = pathEQT+'/mseed_predictor.py'
531 os.system(fullpath2+" -I %s -O %s -F %s" % (infile, outfile, pathEQT))
--> 532 gpd_pick_add(dbsession=session,fileinput=fileinassociate,inventory=inv)
533 else:
534 picker = fbpicker.FBPicker(t_long = 5, freqmin = 1, mode = 'rms', t_ma = 20, nsigma = 7, t_up = 0.7, nr_len = 2, nr_coeff = 2, pol_len = 10, pol_coeff = 10, uncert_coeff = 3)

/usr/local/Caskroom/miniconda/base/envs/easyquake/lib/python3.7/site-packages/easyQuake/easyQuake.py in gpd_pick_add(dbsession, fileinput, inventory)
343 def gpd_pick_add(dbsession=None,fileinput=None,inventory=None):
344 filepath = fileinput
--> 345 with open(filepath) as fp:
346 line = fp.readline()
347 cnt = 1

FileNotFoundError: [Errno 2] No such file or directory: 'data/ba2/20140820/gpd_picks.out'

The code I am using is very similar to your Idaho example but changing latitude longitude and timespan, as well as the machine picker. The code is pasted below.

Can you help solve this?

Would adding an exception to the gpd_pick_add definition before file path = file input solve this issue?

Thank you very much,

Mateo

Code:
`from easyQuake import download_mseed
from easyQuake import daterange
from datetime import date
from easyQuake import combine_associated
from easyQuake import detection_continuous
from easyQuake import association_continuous

from easyQuake import magnitude_quakeml
from easyQuake import simple_cat_df

import matplotlib.pyplot as plt
maxkm = 300 #100
maxdist=300 #100
lat_a = 37.5 #37.8
lat_b = 38.5 #38.34
lon_a = -123 #-122.4
lon_b = -122 #-122.0

start_date = date(2014, 8, 20)
end_date = date(2014, 8, 27)

project_code = 'bay_area'
project_folder = 'data/ba2'
for single_date in daterange(start_date, end_date):
print(single_date.strftime("%Y-%m-%d"))
dirname = single_date.strftime("%Y%m%d")
download_mseed(dirname=dirname, project_folder=project_folder, single_date=single_date, minlat=lat_a, maxlat=lat_b, minlon=lon_a, maxlon=lon_b)
detection_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, single_date=single_date, machine=True, machine_picker= 'EQTransformer', local=True)
association_continuous(dirname=dirname, project_folder=project_folder, project_code=project_code, maxdist=maxdist, maxkm=maxkm, single_date=single_date, local=True)

cat, dfs = combine_associated(project_folder=project_folder, project_code=project_code)
cat = magnitude_quakeml(cat=cat, project_folder=project_folder,plot_event=True)
cat.write('catalog_bayarea_end_08_14.xml',format='QUAKEML')

catdf = simple_cat_df(cat)
plt.figure()
plt.plot(catdf.index,catdf.magnitude,'.')`

Not an issue but a question

Dear Jake,
I am doing some test with your easyQuake suite. It is a very nice "integrated" solution.
I am facing an issue about the location codes of the seismic stations. My network uses location codes (ie 00) but it seems that easyQuake suite drops the location code, and in the catalog produced all the station with pick have location empty location code.
This quite an issue when using after the catalog produced to review the event (with Seiscomp in my case).
Is there an easy way to modify the code ?
Or should I go deep inside to make the modification.
I encountered the same issue with EQT (I am also testing it in standalone mode), and I have had to make some modification in the associator.py script to take the location code in account.
Maybe, if it not easily faisible with the actual easyquake code you may give me some inputs on where I have to search in the code to use the location code of the stations ?

Hope my message is ok, maybe not really clear with my french-english...

Thanks in advance for you answer.
Regards,
Mickael.

hypoinverse depth update

I tried running locate_hyp2000 for depth correction but cat. write command can not update the depth of the event in the XML file.

velocity model, hash and fine tuning

Hi dear @jakewalter and all,

We have installed EasyQuake and run test data. Thank you very much for this great code!
We are planning to use EasyQuake in a local network. I have a few questions regarding this.

  • Is there a detailed document about EasyQuake?
  • We want to use our own velocity model. Where is the velocity model located in the EasyQuake code?
  • Is it possible to fine tune Earthquake Transformer, GPD and PhaseNet for our local network? Where can we do that in the EasyQuake code?
  • How can we use hash? Where are the outputs in the EasyQuake code?

Best regards..

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.