ut-chg / bet Goto Github PK

View Code? Open in Web Editor NEW

11.0 11.0 21.0 20.54 MB

Python package for data-consistent stochastic inverse and forward problems.

Home Page: http://ut-chg.github.io/BET

License: Other

Python 100.00%

bet's Introduction

Computational Hydraulics Group at The Oden Institute for Computational Engineering and Sciences Official Website

This website is developed from the template Alembic. Thank you!

Live at https://ut-chg.github.io.

development guide

Update Website Content: commit to the main branch.

Basic writing and formatting syntax

How to add news?

Create a new markdown file under /_post folder, name it in the date-title.md format (e.g. 2023-01-01-firstpost.md)
In the markdown file, include the follow content.

---
title: your title
categories:
- Workshop(or any other categories)
feature_image: "https://flodismod.github.io/assets/cover_photo.png" (the image on the top of the page)
---

On the News & Outreach page, there is a list of posts. Here you put the preview text, which will be shown below the title of this post.
<!-- more -->

Maintext
<br>

{% include button.html text="Back" link="/news/" %} (this is the button to take you back to news & outreach page)

How to add new group member?

Go to /_includes, find profile_card.html.
Add the following block after the last card:

<div class="card">
    <div class="box2"><img src="/assets/team_profile/new_member.png" alt="Avatar" style="width:100%"></div>
    <div>
        <h4><b>FirstName  LastName</b></h4> 
        <p></p>
        <p class="d">role, Institute</p>
        </div>

    <div class="bottomWrapper">      
        <p ><a href="../ourteam/profile_NewMember">view bio</a></p>
    </div>
</div>

To avoid error, make sure you don't put ".md" in the href of <p ><a href="../ourteam/profile_NewMember">view bio</a></p>.

Go to /ourteam, create a new markdown file profile_NewMember.md, and put the following information down:

---
feature_text: |
  ## Team

feature_image: "/assets/cover_photo.png"
---
{% include profile-short.html name="" extra="" position="" position2=""  bio="" link="/assets/team_profile/"%}

[name of the web link](the actual link)

{% include button.html text="Back" link="/groups/" %}

How to add new research topic?

Go to /_includes, find display_research.html.
Add the following block after the last card following the example:

<div class="card">
  <a href="/project/adaptive_mesh"><img src="/assets/project/melioidosis.png" alt="Avatar" style="width:100%"></a>
    <div class="bottomWrapper">
      <a href="/project/adaptive_mesh"><h6><b>Improving the Efficiency of Wave and Surge Models via Adaptive Mesh Resolution</b></h6></a> 
    </div>
</div>

Go to /project, create a new markdown file adaptive_mesh.md. In the markdown file, put the following information down:

---
title: Title of your project
feature_text: |
  Research

feature_image: "/assets/cover_photo/pier.png" (save the project cover photo in the same directory and quote it here)
feature_credit: Photo by someone (leave it blank if no photographers)

---
<br />

maintext



{% include button.html text="Back" link="/research_page/" %}

How to edit the navigation bar?

Go to _config.yml and edit section under # 9. Site navigation.

bet's People

Contributors

Stargazers

Watchers

bet's Issues

Update the version number in the documentation (and rebuild documentation)

Port features from PolySim (adaptive_sampling)

Port stable features from polysim.run_framework.adaptive_sampling into bet.sampling.adaptiveSampling. Eventually, the adaptive sampling module in PolySim should be completely removed and examples/data in hershel_bin/nonsubdomain/adaptive and the corresponding scripts in lonestar_landuse_bin should reflect this change.

Fix future warning for comparisions

When checking to see if an object is None object == None use type(object) == NoneType instead.

Update documentation of ADCIRC examples

See todos in *.rst files.

plotDomains tests take too long to run

Adjust the plotDomains tests so that they run in seconds instead of minutes.

Typo

We will "wailk" through the following example. This
http://ut-chg.github.io/BET/examples/example_rst_files/validation.html#validation

Add the ability to use arbitrary Ansatzes to calculateP

Documentation

Write it.

Links in example .rst files

Change the links in the example .rst files to point to the github repo for the example rather than download the file.

Testing: adaptiveSampling

Need to make sure *.mat files from parallel and serial runs are getting properly removed in the tearDown method

Move things out of vis that shouldn't be there, like mpi4py stuff.

Remake documentation

The documentation needs to be remade.

Testing: plotP

Tests for bet.postProcess.plotP fail when run in parallel.

Add method to create uniform hyperrectangle

Add way to create uniform hyperrectangle by just specifying the dimensions of that rectangle.

marginal plots

Plotting 2d and then 1d (or reversing the order) marginals will combine the plots in the last saved files.

Update version number in appropriate places

Forward Problem

Add forward problem capabilities and remove the references to a specific \D or \Lambda space and just say metric space.

Dimensions flipped in adaptiveSampling

In line 348 the data arrays are concatenated along the 0-axis, but it should be done along the 1-axis. This may mess up the fromMap example, so I didn't want to change it yet.

Write loadBalance, lbPADCIRC, etc.

Write model running rig that takes care of load balancing within a single TACC(or other HPC) job allocation. We should use mpi4py and maybe look into Pylauncher by TACC. This should take care of running both serial and parallel jobs in parallel. It might make sense to use GNU parallel (but it probably doesn't make sense). Pushkar is interested in this so we should include him.

emulate_iid_lebesgue re-defines num_l_emulate incorrectly

In line 35 of calculateP in the emulate_iid_lebesgue method:

num_l_emulate = int(num_l_emulate/size)+1

Apparently size comes from bet.Comm and is the number of processors used in the computation, so when this operation is called with only a single processor it creates 1 more emulated sample than called for by num_l_emulate parameter.

I came across this while creating the verification example to re-create a uniform distribution on \Lambda and worked around it by passing num_l_emulate-1 into this method. I think the fix is to compute

remainder = num_l_emulate % size

and then for the first "remainder" number of processors add 1 to num_l_emulate. I am not quite sure how to code this though.

Put copyrights on tops of files.

Add ability to sort for low probablity tails

Possible bad parallelism in adaptiveSampling

There is some troubling parallelism in adaptiveSampling.sampler.generalized_chains regarding all_step_ratios. Especially the lines :
MYall_step_ratios = np.copy(all_step_ratios)
all_step_ratios = np.empty((self.num_chains, self.chain_length), dtype=np.float64)
comm.Allgather([MYall_step_ratios, MPI.DOUBLE], [all_step_ratios, MPI.DOUBLE])

In this process, all_step_ratios is being broadcast into a different shape. We need to be careful about the ordering here, I think. The order of entries in all_step_ratios is different after this than doing numpy.reshape. Tests were failing without mpi4py because of this.

I put a temporary fix in the tests_and_parallel branch. I would like @lcgraham to look over this and make sure that there are no errors, and that my fix for the case of no mpi4py is valid. It does make all of the test pass. If you do make changes, please do it in the tests_and_parallel branch, to avoid conflicts when that is merged with master, which should be done very soon.

Add sensitivity package

Finish creating sensitivity package. Create master-sensitivity branch from dev-sensitivity. Remove developmental code from master-sensitivity. Merge master-sensitivity into master. Update documentation. Release as version 0.3 or version 1.0 if all examples are finished.

Verify Calculate P

The solutions, P, from prob,prob_samples_mc, prob_samples_qhull need to be compared to the solutions from the corresponding MATLAB codes Q_singleV, Q_singleVmc, and Q_singleVex (make sure the bin_sizes are equivalent). The bin_size in the MATLAB code is 1/bin_size in the Python code.

Backward compatibility broken in postTools

Be careful with old code using postTools. Recent additions by @lcgraham break backward compatibility. I didn't realize this when I merged it in because the tests were rewritten for the new syntax. I am going to make this more user friendly, but I guess I'll follow the new form.

Add hotstarting to adaptive sampling

Tangled Dimensions

Currently the dimensions for samples in the parameter space (\Lambda) and samples in the data space (D) are:

param, lambda -- (ndim, num_samples)
data -- (num_samples, mdim)

We desire the dimensions of samples from both spaces to match (num_samples, dim). Possible modules this needs to be fixed in are:

adaptiveSampling
basicSampling
simpleFunP
calculateP
plot_D_2D

make sure examples work

Make sure examples work with changes to input and output arguments.

Invaild Divide

BET/bet/calculateP/calculateP.py:262: RuntimeWarning: invalid value encountered in divide
P[Itemp] = rho_D_M[i]*lam_vol[Itemp]/np.sum(lam_vol[Itemp])

Add indicatorFunctions to master

Add indicatorFunctions.py to master from dev branch. Create a class/set of methods that returns the indicator functions and their simple function approximations together since most of these methods take the same arguments. We can create a similar set up in sampling on the dev branch for the gradient based adaptive samplers.

Bring dev up to date with master

Merge master into dev without clobbering developmental portions of dev that differ from master. This is to keep dev in line with the current state of master since there were bugs/issues that were resolved with the addition of the sensitivity package to master.

top_percentile=1.0 returns all samples, postTools.sample_highest_prob

In postTools.sample_highest_prob, if you set top_percentile=1.0, it returns the correct volume of the region in Lambda, but it return num_samples= all samples. It looks like this is because in line 63 postTools

num_samples = np.sum(P_sum <= top_percentile)

So Psum is always <= 1.0, and it returns all samples. Maybe <= top_percentile - epsilon, or an

if top_percentile==1.0
num_samples = numbers of nonzeros in P_samples

Big issues with changes parallelism

Allreduce calculations for integers are all wrong. They are begin treated as numpy arrays of integers which is not giving an error but is doing something completely wrong. Should be switched to allreduce. Many parts of simpleFunP and calculateP are affected.

Update web documentation to reference sensitivity package

Update parallel, overview, and examples to include information about sensitivity package.

Tests_and_parallel

@smattis is the tests_and_parallel branch ready for merging? If so want me to go ahead and merge it to master?

PlotDomains

Update plotDomains so that the dimensions of Q_ref (and similar variables) are automatically fixed if necessary. Automatically create a figs folder if it doesn't already exist and if a custom file name is not specified.

docstrings and comments lacking in plotDomains

The docstrings are not very descriptive in plotDomains, and there are almost no comments. Variable names are not descriptive enough for not having commments. Also, default input arguments should be added to many of the functions.

Change default colormaps in plotDomains so that they are colorblind safe

Verify Calculate P

Add a test to verify that prob_emulated(samples, data, rho_D_M, d_distr_samples) = prob(samples, data, rho_D_M, d_distr_samples)

heurstics

Properly reclass the heuristics in adaptiveSampling so that they all inherit from the heuristic class.

Copyright update

Change copyright at the tops of files to "The BET Development Team" instead of "Lindley Graham and Steven Mattis".

Robustize center_pts_per_edge

Check methods that use center_pts_per_edge. Update these methods so that the argument can be either an int or a list(). If the method receives a list check it to make sure the length matches the dimension of data. If the method receives a int, expand it into an appropriately sized array.