Coder Social home page Coder Social logo

lidongzh / fwiflow.jl Goto Github PK

View Code? Open in Web Editor NEW
39.0 39.0 20.0 12.17 MB

Elastic Full Waveform Inversion for Subsurface Flow Problems Using Intrusive Automatic Differentiation

License: MIT License

Shell 0.94% Julia 99.06%
automatic-differentiation fwi multi-phase-flow

fwiflow.jl's People

Contributors

andrea2333 avatar kailaix avatar lidongzh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

fwiflow.jl's Issues

Permeability inversion

Hi,

Sorry to ask to questions at a time.
As I mentioned in my previous question, I am using your code on this website
https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/flow/

I also read your paper about Physics Constrained Learning (you reference it on the website above). You explain in the paper that you use a neural network to predict the permeabilities and you also present results (in the paper) that look the same as the ones you the website. When I run your code I do get the same results, but I couldn't find a neural network in your code.
Is the neural network built into one of the functions of FwiFlow?

Compile issue on linux machine

Hi developers,

First, thank you for making this open-source software available, which helped me huge on my own research progress.

I ran into a compile issue on linux machine. The error log is shown below

(v1.3) pkg> build FwiFlow
  Building Conda ── `~/.julia/packages/Conda/x5ml4/deps/build.log`
  Building PyCall ─ `~/.julia/packages/PyCall/tqyST/deps/build.log`
  Building HDF5 ─── `~/.julia/packages/HDF5/T1b9x/deps/build.log`
  Building Dierckx  `~/.julia/packages/Dierckx/ZTanR/deps/build.log`
  Building FFTW ─── `~/.julia/packages/FFTW/ayqyZ/deps/build.log`
  Building ADCME ── `~/.julia/packages/ADCME/IQQrn/deps/build.log`
┌ Error: Error building `ADCME`: 
│ Collecting package metadata (current_repodata.json): ...working... done
│ Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
│ Solving environment: ...working... failed with repodata from current_repodata.json, will retry with next repodata source.
│ Collecting package metadata (repodata.json): ...working... done
│ Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Examining @/linux-64::__unix==0=0:  17%|█▋        | 1/6 [00:00<00:00, 9489.38it/Examining adcme:  33%|███▎      | 2/6 [00:00<00:00, 10192.72it/s]               Examining @/linux-64::__archspec==1=x86_64:  67%|██████▋   | 4/6 [00:00<00:00, 5Examining @/linux-64::__glibc==2.17=0:  83%|████████▎ | 5/6 [00:00<00:00, 71.91i                                                                                                                                                          
│ Found conflicts! Looking for incompatible packages.
│ This can take several minutes.  Press CTRL-C to abort.
│ failed
│ 
│ UnsatisfiableError: The following specifications were found
│ to be incompatible with the existing python installation in your environment:
│ 
│ Specifications:
│ 
│   - adcme -> python=3.7
│ 
│ Your python: python=3.8
│ 
│ If python is on the left-most side of the chain, that's the version you've asked for.
│ When python appears to the right, that indicates that the thing on the left is somehow
│ not available for the python version you are constrained to. Note that conda will not
│ change your python version to a different minor version unless you explicitly specify
│ that.
│ 
│ 
│ 
│ [ Info:  ########### Install Tensorflow Dependencies  ########### 
│ [ Info: Running `conda install -y -c kailaix adcme` in root environment
│ ERROR: LoadError: failed process: Process(setenv(`/home/zyin62/.julia/conda/3/bin/conda install -y -c kailaix adcme`,["PATH=/home/zyin62/GATechBundle/Julia/julia-1.3.1/bin:/home/zyin62/GATechBundle/Julia/julia-1.2.0/bin:/home/zyin62/GATechBundle/GCC/9_2_0/bin:/home/zyin62/GATechBundle/Julia/julia-1.5.3/bin:/home/zyin62/GATechBundle/Miniconda3/bin:/home/zyin62/GATechBundle/ScholarlyMarkdown/bin:/home/zyin62/GATechBundle/bin:/home/zyin62/GATechBundle/ibin:/home/zyin62/.local/bin:/home/zyin62/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/root/bin:/home/zyin62/.local/bin:/home/zyin62/bin", "DEVITO_LANGUAGE=openmp", "GATechBundle=/home/zyin62/GATechBundle", "LD_LIBRARY_PATH=/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/libsos:/home/zyin62/GATechBundle/GCC/9_2_0/lib64:/home/zyin62/GATechBundle/GCC/9_2_0/lib:/home/zyin62/GATechBundle/lib64:/home/zyin62/GATechBundle/lib:/home/zyin62/GATechBundle/Miniconda3/lib", "MATLABPATH=/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/mecv:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct3d/mex:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct_wrapping_cpp/mex:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct_wrapping_matlab:/home/zyin62/GATechBundle/tools/transforms/CurveLabSLIM/fdct_usfft_matlab", "MANPATH=/home/zyin62/GATechBundle/Julia/julia-1.3.1/share/man:/home/zyin62/GATechBundle/Julia/julia-1.2.0/share/man:/home/zyin62/GATechBundle/GCC/9_2_0/share/man:/home/zyin62/GATechBundle/Julia/julia-1.5.3/share/man:/home/zyin62/GATechBundle/Miniconda3/share/man:/home/zyin62/GATechBundle/share/man:/usr/local/share/man:/usr/share/man/overrides:/usr/share/man", "MODULEPATH=/usr/share/Modules/modulefiles:/etc/modulefiles:/home/zyin62/GATechBundle/modulefiles:/home/zyin62/privatemodules", "USER=zyin62", "LD_RUN_PATH=/home/zyin62/GATechBundle/GCC/9_2_0/lib64:/home/zyin62/GATechBundle/GCC/9_2_0/lib", "XDG_DATA_DIRS=/home/zyin62/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share"  …  "BASH_FUNC_module()=() {  eval `/usr/bin/modulecmd bash \$*`\n}", "SSH_CONNECTION=172.16.84.126 61286 10.137.63.231 22", "OMP_NUM_THREADS=4", "HISTCONTROL=ignoredups", "MAIL=/var/spool/mail/zyin62", "CONDA_PREFIX=/home/zyin62/.julia/conda/3", "XDG_SESSION_ID=7621", "JULIA_LOAD_PATH=@:/tmp/jl_WQikU0", "LS_COLORS=rs=0:di=38;5;27:ln=38;5;51:mh=44;38;5;15:pi=40;38;5;11:so=38;5;13:do=38;5;5:bd=48;5;232;38;5;11:cd=48;5;232;38;5;3:or=48;5;232;38;5;9:mi=05;48;5;232;38;5;15:su=48;5;196;38;5;15:sg=48;5;11;38;5;16:ca=48;5;196;38;5;226:tw=48;5;10;38;5;16:ow=48;5;10;38;5;21:st=48;5;21;38;5;15:ex=38;5;34:*.tar=38;5;9:*.tgz=38;5;9:*.arc=38;5;9:*.arj=38;5;9:*.taz=38;5;9:*.lha=38;5;9:*.lz4=38;5;9:*.lzh=38;5;9:*.lzma=38;5;9:*.tlz=38;5;9:*.txz=38;5;9:*.tzo=38;5;9:*.t7z=38;5;9:*.zip=38;5;9:*.z=38;5;9:*.Z=38;5;9:*.dz=38;5;9:*.gz=38;5;9:*.lrz=38;5;9:*.lz=38;5;9:*.lzo=38;5;9:*.xz=38;5;9:*.bz2=38;5;9:*.bz=38;5;9:*.tbz=38;5;9:*.tbz2=38;5;9:*.tz=38;5;9:*.deb=38;5;9:*.rpm=38;5;9:*.jar=38;5;9:*.war=38;5;9:*.ear=38;5;9:*.sar=38;5;9:*.rar=38;5;9:*.alz=38;5;9:*.ace=38;5;9:*.zoo=38;5;9:*.cpio=38;5;9:*.7z=38;5;9:*.rz=38;5;9:*.cab=38;5;9:*.jpg=38;5;13:*.jpeg=38;5;13:*.gif=38;5;13:*.bmp=38;5;13:*.pbm=38;5;13:*.pgm=38;5;13:*.ppm=38;5;13:*.tga=38;5;13:*.xbm=38;5;13:*.xpm=38;5;13:*.tif=38;5;13:*.tiff=38;5;13:*.png=38;5;13:*.svg=38;5;13:*.svgz=38;5;13:*.mng=38;5;13:*.pcx=38;5;13:*.mov=38;5;13:*.mpg=38;5;13:*.mpeg=38;5;13:*.m2v=38;5;13:*.mkv=38;5;13:*.webm=38;5;13:*.ogm=38;5;13:*.mp4=38;5;13:*.m4v=38;5;13:*.mp4v=38;5;13:*.vob=38;5;13:*.qt=38;5;13:*.nuv=38;5;13:*.wmv=38;5;13:*.asf=38;5;13:*.rm=38;5;13:*.rmvb=38;5;13:*.flc=38;5;13:*.avi=38;5;13:*.fli=38;5;13:*.flv=38;5;13:*.gl=38;5;13:*.dl=38;5;13:*.xcf=38;5;13:*.xwd=38;5;13:*.yuv=38;5;13:*.cgm=38;5;13:*.emf=38;5;13:*.axv=38;5;13:*.anx=38;5;13:*.ogv=38;5;13:*.ogx=38;5;13:*.aac=38;5;45:*.au=38;5;45:*.flac=38;5;45:*.mid=38;5;45:*.midi=38;5;45:*.mka=38;5;45:*.mp3=38;5;45:*.mpc=38;5;45:*.ogg=38;5;45:*.ra=38;5;45:*.wav=38;5;45:*.axa=38;5;45:*.oga=38;5;45:*.spx=38;5;45:*.xspf=38;5;45:", "OPENBLAS_MAIN_FREE=1"]), ProcessExited(1)) [1]
│ 
│ Stacktrace:
│  [1] pipeline_error at ./process.jl:525 [inlined]
│  [2] #run#565(::Bool, ::typeof(run), ::Cmd) at ./process.jl:440
│  [3] run at ./process.jl:438 [inlined]
│  [4] runconda(::Cmd, ::String) at /home/zyin62/.julia/packages/Conda/x5ml4/src/Conda.jl:116
│  [5] #add#1(::String, ::typeof(Conda.add), ::String, ::String) at /home/zyin62/.julia/packages/Conda/x5ml4/src/Conda.jl:190
│  [6] (::Conda.var"#kw##add")(::NamedTuple{(:channel,),Tuple{String}}, ::typeof(Conda.add), ::String) at ./none:0
│  [7] top-level scope at /home/zyin62/.julia/packages/ADCME/IQQrn/deps/build.jl:45
│  [8] include at ./boot.jl:328 [inlined]
│  [9] include_relative(::Module, ::String) at ./loading.jl:1105
│  [10] include(::Module, ::String) at ./Base.jl:31
│  [11] include(::String) at ./client.jl:424
│  [12] top-level scope at none:5in expression starting at /home/zyin62/.julia/packages/ADCME/IQQrn/deps/build.jl:44
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649
  Building FwiFlow  `~/.julia/packages/FwiFlow/ynbyK/deps/build.log`
┌ Error: Error building `FwiFlow`: 
│ which: no nvcc in (/home/zyin62/GATechBundle/Julia/julia-1.3.1/bin:/home/zyin62/GATechBundle/Julia/julia-1.2.0/bin:/home/zyin62/GATechBundle/GCC/9_2_0/bin:/home/zyin62/GATechBundle/Julia/julia-1.5.3/bin:/home/zyin62/GATechBundle/Miniconda3/bin:/home/zyin62/GATechBundle/ScholarlyMarkdown/bin:/home/zyin62/GATechBundle/bin:/home/zyin62/GATechBundle/ibin:/home/zyin62/.local/bin:/home/zyin62/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/usr/local/root/bin:/home/zyin62/.local/bin:/home/zyin62/bin)
│ ERROR: LoadError: ADCME is not properly built; run `Pkg.build("ADCME")` to fix the problem.
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] top-level scope at /home/zyin62/.julia/packages/ADCME/IQQrn/src/ADCME.jl:36
│  [3] include at ./boot.jl:328 [inlined]
│  [4] include_relative(::Module, ::String) at ./loading.jl:1105
│  [5] include(::Module, ::String) at ./Base.jl:31
│  [6] top-level scope at none:2
│  [7] eval at ./boot.jl:330 [inlined]
│  [8] eval(::Expr) at ./client.jl:425
│  [9] top-level scope at ./none:3in expression starting at /home/zyin62/.julia/packages/ADCME/IQQrn/src/ADCME.jl:33
│ ERROR: LoadError: Failed to precompile ADCME [07b341a0-ce75-57c6-b2de-414ffdc00be5] to /home/zyin62/.julia/compiled/v1.3/ADCME/b8Ld2_0cxUW.ji.
│ Stacktrace:
│  [1] error(::String) at ./error.jl:33
│  [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
│  [3] _require(::Base.PkgId) at ./loading.jl:1024
│  [4] require(::Base.PkgId) at ./loading.jl:922
│  [5] require(::Module, ::Symbol) at ./loading.jl:917
│  [6] include at ./boot.jl:328 [inlined]
│  [7] include_relative(::Module, ::String) at ./loading.jl:1105
│  [8] include(::Module, ::String) at ./Base.jl:31
│  [9] include(::String) at ./client.jl:424
│  [10] top-level scope at none:5in expression starting at /home/zyin62/.julia/packages/FwiFlow/ynbyK/deps/build.jl:10
│   Updating registry at `~/.julia/registries/SLIMregistryJL`
│   Updating git-repo `https://github.com/slimgroup/SLIMregistryJL.git`
   Resolving package versions...
│   Updating `/tmp/jl_j9YNnc/Project.toml`
│  [no changes]
│   Updating `/tmp/jl_j9YNnc/Manifest.toml`
│   [621f4979]  AbstractFFTs v0.5.0  v1.0.0
│   [7a1cc6ca]  FFTW v1.3.0  v1.3.1
│   [f67ccb44]  HDF5 v0.13.6  v0.13.7
│   [0234f1f7]  HDF5_jll v1.12.0+1  v1.10.5+7
│   [deac9b47] - LibCURL_jll v7.70.0+2
│   [29816b5a] - LibSSH2_jll v1.9.0+3
│   [c8ffd9c3] - MbedTLS_jll v2.16.8+1
│   [458c3c95] - OpenSSL_jll v1.1.1+6
│   [8e850ede] - nghttp2_jll v1.40.0+2
│   [2a0f44e3] ~ Base64  [`@stdlib/Base64`]  
│   [ade2ca70] ~ Dates  [`@stdlib/Dates`]  
│   [8bb1440f] ~ DelimitedFiles  [`@stdlib/DelimitedFiles`]  
│   [8ba89e20] ~ Distributed  [`@stdlib/Distributed`]  
│   [b77e0a4c] ~ InteractiveUtils  [`@stdlib/InteractiveUtils`]  
│   [76f85450] ~ LibGit2  [`@stdlib/LibGit2`]  
│   [8f399da3] ~ Libdl  [`@stdlib/Libdl`]  
│   [56ddb016] ~ Logging  [`@stdlib/Logging`]  
│   [d6f4376e] ~ Markdown  [`@stdlib/Markdown`]  
│   [a63ad114] ~ Mmap  [`@stdlib/Mmap`]  
│   [44cfe95a] ~ Pkg  [`@stdlib/Pkg`]  
│   [de0858da] ~ Printf  [`@stdlib/Printf`]  
│   [3fa0cd96] ~ REPL  [`@stdlib/REPL`]  
│   [ea8e919c] ~ SHA  [`@stdlib/SHA`]  
│   [9e88b42a] ~ Serialization  [`@stdlib/Serialization`]  
│   [1a1011a3] ~ SharedArrays  [`@stdlib/SharedArrays`]  
│   [6462fe0b] ~ Sockets  [`@stdlib/Sockets`]  
│   [2f01184e] ~ SparseArrays  [`@stdlib/SparseArrays`]  
│   [10745b16] ~ Statistics  [`@stdlib/Statistics`]  
│   [8dfed614] ~ Test  [`@stdlib/Test`]  
│   [cf7118a7] ~ UUIDs  [`@stdlib/UUIDs`]  
│   [4ec0a83e] ~ Unicode  [`@stdlib/Unicode`]  
└ @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Pkg/src/backwards_compatible_isolation.jl:649

But I used to work on this code in my Mac laptop, where everything works well. I gave it a look and found out that some dynamic libraries fail to compile well, e.g.

[zyin62@eas-coda-fherr08 build]$ pwd
/home/zyin62/.julia/packages/FwiFlow/ynbyK/deps/CustomOps/Upwlap/build
[zyin62@eas-coda-fherr08 build]$ ls
libUpwlapOp.dylib

This library is in dylib format, where linux cannot read. This ends up to be an error when I run

out_sw_true, out_p_true = imseq(tfCtxTrue)

in your example here https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/flow/

Can you offer me some help on this? Thank you very much!!

Ziyi (Francis) Yin

Convergence analysis

Hi,

I really like your work on FwiFlow and ADCME. I am currently using your simulation code you showcase on the link below as a bases to produce data that I use to train some neural networks. Do you happen to have a convergence analysis of this simulation? I am asking because my neural networks keep on convergence to something very similar but slidely off. I am wondering now if this could be an error inside my code or if the simulation may just not be accurate enough, meaning I would need to use finer grid.

Link:
https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/flow/

Any hint would be appreciated.

Niko

Problems during execution of the program

Hi developer

Thanks for this great contribution. Now I am trying to run the offered examples, however, always encounter some problems as follows.


  1. I cannot find where is test_data.mat as needed in docs/src/tutorials/fwi.md.
  2. I am not sure what is the current name of Ops/AdvectionDiffusion/ as described in some examples.
  3. I cannot find the location of generate_m.jl for generating data.
  4. When testing the example FwiFlow-0.3.1.jl/docs/codes/src_fwi_channel, I do not know the correct order to run the example. But also find some problems in data:
  • For main_mar_fwi.jl, cannot find FWI/Src/params/Mar_source_2001.bin.
  • For main_run.py or main_two_phase_flow_inversion.jl, there is an error of TypeError("Tensors in list passed to 'values' of 'Pack' Op have types [int32, <NOT CONVERTIBLE TO TENSOR>, <NOT CONVERTIBLE TO TENSOR>] that don't all match.") at out_sw_init, out_p_init = imseq(tfCtxInit)
  1. For CustomOps/FWI
  • For CustomOps/FWI/Obcalc.jl, cannot find ./params/ricker_10Hz.bin.
  • For CustomOps/FWI/main_calc_obs_data.py, cannot find ../../AdvectionDiffusion/data/U1.txt`.
  • For FwiFlow-0.3.1.jl/deps/CustomOps/FWI/paraGene.py, the names 'Cp, Cs, Den, nz, nx' are not defined.

I have attempted to find these files in different versions of FwiFlow.jl and also TwoPhaseFlowFWI, but not successful.

I hope I could get your help.

Thanks!

dylib not found

Hi Dongzhuo and Kailai,

Greetings! I am using your software FwiFlow.jl to reproduce your FWI tutorial in https://lidongzh.github.io/FwiFlow.jl/dev/tutorials/fwi/ but meet the error as below.

I am on a Mac system and julia1.3 version. I ran the examples in ADCME tutorials and they are fine. I would really appreciate your help on this!

Also, do you have any example for FWI-Two Phase Flow Coupled Inversion, e.g. in your paper? I didn't find any in your tutorial webpage in https://lidongzh.github.io/FwiFlow.jl/dev/.

Thank you! Hope you both enjoyed your Thanksgiving holiday.

Cheers,

Ziyi (Francis) Yin

julia> obs = compute_observation(sess, fwi, cp, cs, ρ, stf, shot_ids, gpu_id=0) ERROR: File /Users/francisyin/.julia/packages/FwiFlow/ynbyK/deps/CustomOps/FWI/build/libFwiOp.dylib not found. If you intend to load a library by absolute path, try tf.load_op_library(filename). Stacktrace: [1] error(::String) at ./error.jl:33 [2] load_library(::String) at /Users/francisyin/.julia/packages/ADCME/JzSON/src/extra.jl:1002 [3] #load_op#247(::Missing, ::typeof(load_op), ::String, ::String) at /Users/francisyin/.julia/packages/ADCME/JzSON/src/extra.jl:141 [4] load_op at /Users/francisyin/.julia/packages/ADCME/JzSON/src/extra.jl:136 [inlined] [5] fwi_obs_op(::PyCall.PyObject, ::PyCall.PyObject, ::PyCall.PyObject, ::PyCall.PyObject, ::Int64, ::PyCall.PyObject, ::String) at /Users/francisyin/.julia/packages/FwiFlow/ynbyK/src/Core.jl:51 [6] #compute_observation#17(::Int64, ::typeof(compute_observation), ::PyCall.PyObject, ::FWI, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,1}, ::Array{Int64,1}) at /Users/francisyin/.julia/packages/FwiFlow/ynbyK/src/FWI.jl:115 [7] (::FwiFlow.var"#kw##compute_observation")(::NamedTuple{(:gpu_id,),Tuple{Int64}}, ::typeof(compute_observation), ::PyCall.PyObject, ::FWI, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Float64,1}, ::Array{Int64,1}) at ./none:0 [8] top-level scope at REPL[34]:1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.