geoscienceaustralia / hiqga.jl

High Quality Geophysical Analysis provides a general purpose Bayesian and deterministic inversion framework for various geophysical methods and spatially distributed / timeseries data

HiQGA.jl

This is a general purpose package for spatial statistical inference, geophysical forward modeling, Bayesian inference and inversion (both deterministic and probabilistic).

Readily usable geophysical forward operators are for AEM, CSEM and MT physics (references underneath), for which the time domain AEM codes are fairly production-ready. We've added SMR physics too! The current EM modeling is in 1D, but the inversion framework is dimensionally agnostic (e.g., you can regress images). Adding your own geophysical operators is easy!

Installation

To install the latest stable release, in a perfect world we'd use Julia's Pkg REPL by hitting ] to enter pkg> mode. Then enter the following, at the pkg> prompt:

pkg> add HiQGA

For the latest development version on here, you'd want to then do

pkg> dev HiQGA

Docs

References, detailed instructions for installation, running examples and setting your environment on a cluster are ☞

Example AEM inversion

hiqga.jl's Issues

Discussion on capturing required functionalities within package

Following an early discussion with Yusen, I captured a few items as follows that it is better to be discussed in a meeting.

• Having the read_survey_file function as a more generic function that deals with any type of input dataset. May be similar approach in our previously developed scripts could be adopted here, i.e. *.stm file that basically extracts important information from input data to feed into inversion code. In this way, we can handle both SkyTEM, Tempest etc. in a harmonised approach.
• As an open-source code, the analyst/developer might need to add/change a few components of the function. The current version is structured in a way that it is a bit difficult to navigate to where a particular function is located.
• Two plots as shown here are not representative enough. For instance, the high altitude noise for HM and LM is better to be displayed in a graph way rather than what the left panels show. Having observed and modelled data to better understand the misfit definitely helps.
  
  

minimize writes from mpi runs

• in gradbased, show only last iteration
• in mcmc, don't show every 1000 if opt.showstats == false

compress deterministic inversion results into single file

will save inode count tremendously

Geometry as fixed values instead of column numbers for SkyTEM

For now, the fix is the following [dx dy dz] columns in Ross' convention for SkyTEM.

using DelimitedFiles
A = readdlm("L103004.XYZ")
B = [A [-13.32  0.0 2.0].*ones(size(A, 1))]
writedlm("new.txt", B)

Quit on no improvement in `GradientInversion.jl`

use pmap for processing SkyTEM or Tempest transD plots

• We should be able to process all soundings for summary images independent of each other.
• If refactoring code to cat all outputs into one file, cat refactor needs to be done first @richardt94

Delauny outputs from HiQGA

Would like to implement a Delaunay triangulation from point clouds, and visualise this within ParaView as a VTK.

Can we make a button for the docs

And link it in README.md ?

plot_posterior pmap graceful failure

use on_error=ex->a where a is a vector of stuff to cat and reduce

read_survey_file for Tempest: use strings

same as

HiQGA.jl/src/SkyTEM1DInversion.jl

Line 172 in fc3b4e1

function read_survey_files(dfnfile::String;

• move getcolNo() getcolNo_except() and getlocationinhdr() into CommonToAll and do not replicate

make splits in AEMnoNuisanceMcMC

structs for scripts with functions

Instead of re-running scripts for plotting, load them through a struct

• Will require minimal addition through multiple dispatch of functions to accept structs,
• dispatch can be used to call the old functions with arguments and keywords.
• Can use serialize / deserialze? See here

Vertical exaggeration calculation

• make a function to calculate vertical exaggeration for 2D plots
• add it to Tempest prob, SkyTEM prob and SkyTEM deterministic inv 2D sections

Refactor common names for functions

no nuisance:
summaryAEMimages, plotindividualAEMsoundings
and
with nuisance
summaryimages, plotindividualsoundings
just like
loopacrossAEMsoundings
the number of inputs is usually different so we can get away with a definition in AbstractOperator.jl and an import in the relevant modules, followed by an export in those or in AbstractOperator itself.

may be a breaking change, can be fixed with an oldname=newname

PDF of fits from all lines in SkyTEM

Phi_d 0-1.1, 1.1-2, and greater than 2 in three bins are now plotted according to commit 3c8cd71

Improve plots for deterministic inversions

• use the add dummy subplot trick to add rows as in

  HiQGA.jl/src/CommonToAll.jl

  Line 1964 in 2517259

  function plotmanygrids(σ, X, Y, Z, zall; yl=[], xl=[],

• add in PLNI and sferics to soundings, do not plot if they are not provided, ingest as array of nothing in soundings

Write ASEG-GDF files for deterministic inversions

Can do by using

HiQGA.jl/src/CommonToAll.jl

Line 1893 in 2517259

function readfzipped(fzipped::String, nlayers::Int; nnu=0)

and writing out appropriate headers

[Bug] constructor for `OptionsNuisance` modifies its own arguments

The function here takes an array argument sdev which is then bound to a new OptionsNuisance struct. On this line the array is modified in-place (because it is supposed to represent the width of the proposal distribution as a proportion of the prior bounds on a nuisance). Because array arguments in Julia are passed by reference, this means that the sdev array which is passed to the constructor function is silently changed by the function - this is an issue if, for example, an inversion is being performed with the same sdev parameters for a number of separate soundings/datasets as those parameters will be silently modified every time an inversion is initialised.

make splits in gradient based AEM plots if longer than Rmax

Read ASEG-GDF description file

Generate hdr file like GA-AEM produces which looks like

1   fiducial
2   line
3   flight
4   datetime
5   date
6   utc_time
.
.
.

Make HDR file for deterministic outputs

Make sure to write headers if none exist

Plot specific sounding results for deterministic inversions

You'll probably get this from a line image, so

• Find line in file (use headers for this, first fix #54 )
• Calculate distance along line
• get conductivity from the row
• get nu from the same row if needed

TEMPEST amplitude inversion plots

Right now we still plot against Bx and Bz when we should plot against sqrt.(Bx.^2 + Bz.^2)

make line distances cumulative before plotting

see some where around here