lidongzh / fwiflow.jl Goto Github PK
View Code? Open in Web Editor NEWElastic Full Waveform Inversion for Subsurface Flow Problems Using Intrusive Automatic Differentiation
License: MIT License
Elastic Full Waveform Inversion for Subsurface Flow Problems Using Intrusive Automatic Differentiation
License: MIT License
Hi,
Sorry to ask to questions at a time.
As I mentioned in my previous question, I am using your code on this website
https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/flow/
I also read your paper about Physics Constrained Learning (you reference it on the website above). You explain in the paper that you use a neural network to predict the permeabilities and you also present results (in the paper) that look the same as the ones you the website. When I run your code I do get the same results, but I couldn't find a neural network in your code.
Is the neural network built into one of the functions of FwiFlow?
Hi developers,
First, thank you for making this open-source software available, which helped me huge on my own research progress.
I ran into a compile issue on linux machine. The error log is shown below
(v1.3) pkg> build FwiFlow
Building Conda ──→ `~/.julia/packages/Conda/x5ml4/deps/build.log`
Building PyCall ─→ `~/.julia/packages/PyCall/tqyST/deps/build.log`
Building HDF5 ───→ `~/.julia/packages/HDF5/T1b9x/deps/build.log`
Building Dierckx → `~/.julia/packages/Dierckx/ZTanR/deps/build.log`
Building FFTW ───→ `~/.julia/packages/FFTW/ayqyZ/deps/build.log`
Building ADCME ──→ `~/.julia/packages/ADCME/IQQrn/deps/build.log`
┌ Error: Error building `ADCME`:
│ Collecting package metadata (current_repodata.json): ...working... done
│ Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
│ Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
│ Collecting package metadata (repodata.json): ...working... done
│ Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Examining @/linux-64::__unix==0=0: 17%|█▋ | 1/6 [00:00<00:00, 9489.38it/Examining adcme: 33%|███▎ | 2/6 [00:00<00:00, 10192.72it/s] Examining @/linux-64::__archspec==1=x86_64: 67%|██████▋ | 4/6 [00:00<00:00, 5Examining @/linux-64::__glibc==2.17=0: 83%|████████▎ | 5/6 [00:00<00:00, 71.91i
│ Found conflicts! Looking for incompatible packages.
│ This can take several minutes. Press CTRL-C to abort.
│ failed
│
│ UnsatisfiableError: The following specifications were found
│ to be incompatible with the existing python installation in your environment:
│
│ Specifications:
│
│ - adcme -> python=3.7
│
│ Your python: python=3.8
│
│ If python is on the left-most side of the chain, that's the version you've asked for.
│ When python appears to the right, that indicates that the thing on the left is somehow
│ not available for the python version you are constrained to. Note that conda will not
│ change your python version to a different minor version unless you explicitly specify
│ that.
│
│
│
│ [ Info: ########### Install Tensorflow Dependencies ###########
│ [ Info: Running `conda install -y -c kailaix adcme` in root environment
│ ERROR: LoadError: failed process: Process(setenv(`/home/zyin62/.julia/conda/3/bin/conda install -y -c kailaix adcme`,["PATH=/home/zyin62/GATechBundle/Julia/julia-1.3.1/bin:/home/zyin62/GATechBundle/Julia/julia-1.2.0/bin:/home/zyin62/GATechBundle/GCC/9_2_0/bin:/home/zyin62/GATechBundle/Julia/julia-1.5.3/bin:/home/zyin62/GATechBundle/Miniconda3/bin:/home/zyin62/GATechBundle/ScholarlyMarkdown/bin:/home/zyin62/GATechBundle/bin:/home/zyin62/GATechBundle/ibin:/home/zyin62/.local/bin:/home/zyin62/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/root/bin:/home/zyin62/.local/bin:/home/zyin62/bin", "DEVITO_LANGUAGE=openmp", "GATechBundle=/home/zyin62/GATechBundle", "LD_LIBRARY_PATH=/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/libsos:/home/zyin62/GATechBundle/GCC/9_2_0/lib64:/home/zyin62/GATechBundle/GCC/9_2_0/lib:/home/zyin62/GATechBundle/lib64:/home/zyin62/GATechBundle/lib:/home/zyin62/GATechBundle/Miniconda3/lib", "MATLABPATH=/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/mecv:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct3d/mex:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct_wrapping_cpp/mex:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct_wrapping_matlab:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct_usfft_matlab", "MANPATH=/home/zyin62/GATechBundle/Julia/julia-1.3.1/share/man:/home/zyin62/GATechBundle/Julia/julia-1.2.0/share/man:/home/zyin62/GATechBundle/GCC/9_2_0/share/man:/home/zyin62/GATechBundle/Julia/julia-1.5.3/share/man:/home/zyin62/GATechBundle/Miniconda3/share/man:/home/zyin62/GATechBundle/share/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man", "MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/home/zyin62/GATechBundle/modulefiles:/home/zyin62/privatemodules", "USER=zyin62", "LD_RUN_PATH=/home/zyin62/GATechBundle/GCC/9_2_0/lib64:/home/zyin62/GATechBundle/GCC/9_2_0/lib", "XDG_DATA_DIRS=/home/zyin62/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share" … "BASH_FUNC_module()=() { eval `/usr/bin/modulecmd bash \$*`\n}", "SSH_CONNECTION=172.16.84.126 61286 10.137.63.231 22", "OMP_NUM_THREADS=4", "HISTCONTROL=ignoredups", "MAIL=/var/spool/mail/zyin62", "CONDA_PREFIX=/home/zyin62/.julia/conda/3", "XDG_SESSION_ID=7621", "JULIA_LOAD_PATH=@:/tmp/jl_WQikU0", "LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:", "OPENBLAS_MAIN_FREE=1"]), ProcessExited(1)) [1]
│
│ Stacktrace:
│ [1] pipeline_error at ./process.jl:525 [inlined]
│ [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
│ [3] run at ./process.jl:438 [inlined]
│ [4] runconda(::Cmd, ::String) at /home/zyin62/.julia/packages/Conda/x5ml4/src/Conda.jl:116
│ [5] #add#1(::String, ::typeof(Conda.add), ::String, ::String) at /home/zyin62/.julia/packages/Conda/x5ml4/src/Conda.jl:190
│ [6] (::Conda.var"#kw##add")(::NamedTuple{(:channel,),Tuple{String}}, ::typeof(Conda.add), ::String) at ./none:0
│ [7] top-level scope at /home/zyin62/.julia/packages/ADCME/IQQrn/deps/build.jl:45
│ [8] include at ./boot.jl:328 [inlined]
│ [9] include_relative(::Module, ::String) at ./loading.jl:1105
│ [10] include(::Module, ::String) at ./Base.jl:31
│ [11] include(::String) at ./client.jl:424
│ [12] top-level scope at none:5
│ in expression starting at /home/zyin62/.julia/packages/ADCME/IQQrn/deps/build.jl:44
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649
Building FwiFlow → `~/.julia/packages/FwiFlow/ynbyK/deps/build.log`
┌ Error: Error building `FwiFlow`:
│ which: no nvcc in (/home/zyin62/GATechBundle/Julia/julia-1.3.1/bin:/home/zyin62/GATechBundle/Julia/julia-1.2.0/bin:/home/zyin62/GATechBundle/GCC/9_2_0/bin:/home/zyin62/GATechBundle/Julia/julia-1.5.3/bin:/home/zyin62/GATechBundle/Miniconda3/bin:/home/zyin62/GATechBundle/ScholarlyMarkdown/bin:/home/zyin62/GATechBundle/bin:/home/zyin62/GATechBundle/ibin:/home/zyin62/.local/bin:/home/zyin62/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/root/bin:/home/zyin62/.local/bin:/home/zyin62/bin)
│ ERROR: LoadError: ADCME is not properly built; run `Pkg.build("ADCME")` to fix the problem.
│ Stacktrace:
│ [1] error(::String) at ./error.jl:33
│ [2] top-level scope at /home/zyin62/.julia/packages/ADCME/IQQrn/src/ADCME.jl:36
│ [3] include at ./boot.jl:328 [inlined]
│ [4] include_relative(::Module, ::String) at ./loading.jl:1105
│ [5] include(::Module, ::String) at ./Base.jl:31
│ [6] top-level scope at none:2
│ [7] eval at ./boot.jl:330 [inlined]
│ [8] eval(::Expr) at ./client.jl:425
│ [9] top-level scope at ./none:3
│ in expression starting at /home/zyin62/.julia/packages/ADCME/IQQrn/src/ADCME.jl:33
│ ERROR: LoadError: Failed to precompile ADCME [07b341a0-ce75-57c6-b2de-414ffdc00be5] to /home/zyin62/.julia/compiled/v1.3/ADCME/b8Ld2_0cxUW.ji.
│ Stacktrace:
│ [1] error(::String) at ./error.jl:33
│ [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
│ [3] _require(::Base.PkgId) at ./loading.jl:1024
│ [4] require(::Base.PkgId) at ./loading.jl:922
│ [5] require(::Module, ::Symbol) at ./loading.jl:917
│ [6] include at ./boot.jl:328 [inlined]
│ [7] include_relative(::Module, ::String) at ./loading.jl:1105
│ [8] include(::Module, ::String) at ./Base.jl:31
│ [9] include(::String) at ./client.jl:424
│ [10] top-level scope at none:5
│ in expression starting at /home/zyin62/.julia/packages/FwiFlow/ynbyK/deps/build.jl:10
│ Updating registry at `~/.julia/registries/SLIMregistryJL`
│ Updating git-repo `https://github.com/slimgroup/SLIMregistryJL.git`
Resolving package versions...
│ Updating `/tmp/jl_j9YNnc/Project.toml`
│ [no changes]
│ Updating `/tmp/jl_j9YNnc/Manifest.toml`
│ [621f4979] ↑ AbstractFFTs v0.5.0 ⇒ v1.0.0
│ [7a1cc6ca] ↑ FFTW v1.3.0 ⇒ v1.3.1
│ [f67ccb44] ↑ HDF5 v0.13.6 ⇒ v0.13.7
│ [0234f1f7] ↓ HDF5_jll v1.12.0+1 ⇒ v1.10.5+7
│ [deac9b47] - LibCURL_jll v7.70.0+2
│ [29816b5a] - LibSSH2_jll v1.9.0+3
│ [c8ffd9c3] - MbedTLS_jll v2.16.8+1
│ [458c3c95] - OpenSSL_jll v1.1.1+6
│ [8e850ede] - nghttp2_jll v1.40.0+2
│ [2a0f44e3] ~ Base64 [`@stdlib/Base64`] ⇒
│ [ade2ca70] ~ Dates [`@stdlib/Dates`] ⇒
│ [8bb1440f] ~ DelimitedFiles [`@stdlib/DelimitedFiles`] ⇒
│ [8ba89e20] ~ Distributed [`@stdlib/Distributed`] ⇒
│ [b77e0a4c] ~ InteractiveUtils [`@stdlib/InteractiveUtils`] ⇒
│ [76f85450] ~ LibGit2 [`@stdlib/LibGit2`] ⇒
│ [8f399da3] ~ Libdl [`@stdlib/Libdl`] ⇒
│ [56ddb016] ~ Logging [`@stdlib/Logging`] ⇒
│ [d6f4376e] ~ Markdown [`@stdlib/Markdown`] ⇒
│ [a63ad114] ~ Mmap [`@stdlib/Mmap`] ⇒
│ [44cfe95a] ~ Pkg [`@stdlib/Pkg`] ⇒
│ [de0858da] ~ Printf [`@stdlib/Printf`] ⇒
│ [3fa0cd96] ~ REPL [`@stdlib/REPL`] ⇒
│ [ea8e919c] ~ SHA [`@stdlib/SHA`] ⇒
│ [9e88b42a] ~ Serialization [`@stdlib/Serialization`] ⇒
│ [1a1011a3] ~ SharedArrays [`@stdlib/SharedArrays`] ⇒
│ [6462fe0b] ~ Sockets [`@stdlib/Sockets`] ⇒
│ [2f01184e] ~ SparseArrays [`@stdlib/SparseArrays`] ⇒
│ [10745b16] ~ Statistics [`@stdlib/Statistics`] ⇒
│ [8dfed614] ~ Test [`@stdlib/Test`] ⇒
│ [cf7118a7] ~ UUIDs [`@stdlib/UUIDs`] ⇒
│ [4ec0a83e] ~ Unicode [`@stdlib/Unicode`] ⇒
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649
But I used to work on this code in my Mac laptop, where everything works well. I gave it a look and found out that some dynamic libraries fail to compile well, e.g.
[zyin62@eas-coda-fherr08 build]$ pwd
/home/zyin62/.julia/packages/FwiFlow/ynbyK/deps/CustomOps/Upwlap/build
[zyin62@eas-coda-fherr08 build]$ ls
libUpwlapOp.dylib
This library is in dylib
format, where linux cannot read. This ends up to be an error when I run
out_sw_true, out_p_true = imseq(tfCtxTrue)
in your example here https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/flow/
Can you offer me some help on this? Thank you very much!!
Ziyi (Francis) Yin
Hi,
I really like your work on FwiFlow and ADCME. I am currently using your simulation code you showcase on the link below as a bases to produce data that I use to train some neural networks. Do you happen to have a convergence analysis of this simulation? I am asking because my neural networks keep on convergence to something very similar but slidely off. I am wondering now if this could be an error inside my code or if the simulation may just not be accurate enough, meaning I would need to use finer grid.
Link:
https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/flow/
Any hint would be appreciated.
Niko
Hi developer
Thanks for this great contribution. Now I am trying to run the offered examples, however, always encounter some problems as follows.
test_data.mat
as needed in docs/src/tutorials/fwi.md
.Ops/AdvectionDiffusion/
as described in some examples.generate_m.jl
for generating data.FwiFlow-0.3.1.jl/docs/codes/src_fwi_channel
, I do not know the correct order to run the example. But also find some problems in data:main_mar_fwi.jl
, cannot find FWI/Src/params/Mar_source_2001.bin
.main_run.py
or main_two_phase_flow_inversion.jl
, there is an error of TypeError("Tensors in list passed to 'values' of 'Pack' Op have types [int32, <NOT CONVERTIBLE TO TENSOR>, <NOT CONVERTIBLE TO TENSOR>] that don't all match.")
at out_sw_init, out_p_init = imseq(tfCtxInit)
CustomOps/FWI
CustomOps/FWI/Obcalc.jl
, cannot find ./params/ricker_10Hz.bin
.CustomOps/FWI/main_calc_obs_data.py, cannot find
../../AdvectionDiffusion/data/U1.txt`.FwiFlow-0.3.1.jl/deps/CustomOps/FWI/paraGene.py
, the names 'Cp, Cs, Den, nz, nx' are not defined.I have attempted to find these files in different versions of FwiFlow.jl and also TwoPhaseFlowFWI
, but not successful.
I hope I could get your help.
Thanks!
Hi Dongzhuo and Kailai,
Greetings! I am using your software FwiFlow.jl to reproduce your FWI tutorial in https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/fwi/ but meet the error as below.
I am on a Mac system and julia1.3 version. I ran the examples in ADCME tutorials and they are fine. I would really appreciate your help on this!
Also, do you have any example for FWI-Two Phase Flow Coupled Inversion, e.g. in your paper? I didn't find any in your tutorial webpage in https://lidongzh.github.io/FwiFlow.jl/dev/.
Thank you! Hope you both enjoyed your Thanksgiving holiday.
Cheers,
Ziyi (Francis) Yin
julia> obs = compute_observation(sess, fwi, cp, cs, ρ, stf, shot_ids, gpu_id=0) ERROR: File /Users/francisyin/.julia/packages/FwiFlow/ynbyK/deps/CustomOps/FWI/build/libFwiOp.dylib not found. If you intend to load a library by absolute path, try tf.load_op_library(filename). Stacktrace: [1] error(::String) at ./error.jl:33 [2] load_library(::String) at /Users/francisyin/.julia/packages/ADCME/JzSON/src/extra.jl:1002 [3] #load_op#247(::Missing, ::typeof(load_op), ::String, ::String) at /Users/francisyin/.julia/packages/ADCME/JzSON/src/extra.jl:141 [4] load_op at /Users/francisyin/.julia/packages/ADCME/JzSON/src/extra.jl:136 [inlined] [5] fwi_obs_op(::PyCall.PyObject, ::PyCall.PyObject, ::PyCall.PyObject, ::PyCall.PyObject, ::Int64, ::PyCall.PyObject, ::String) at /Users/francisyin/.julia/packages/FwiFlow/ynbyK/src/Core.jl:51 [6] #compute_observation#17(::Int64, ::typeof(compute_observation), ::PyCall.PyObject, ::FWI, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,1}, ::Array{Int64,1}) at /Users/francisyin/.julia/packages/FwiFlow/ynbyK/src/FWI.jl:115 [7] (::FwiFlow.var"#kw##compute_observation")(::NamedTuple{(:gpu_id,),Tuple{Int64}}, ::typeof(compute_observation), ::PyCall.PyObject, ::FWI, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,1}, ::Array{Int64,1}) at ./none:0 [8] top-level scope at REPL[34]:1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.