maxwnewcomer / nwtopt Goto Github PK

currently the user has to run a series of command to get NWTOPT up and running. This is not the intended functionality of NWTOPT and the master NWTOPT.py script needs to be fixed and reworked.

HTCondor allows for logging, but the folder specified in nwtopt.sub doesn't exist, thus no logging occurs. Simple fix. Should look like this:

• ./logs/errors
• ./logs/outputs
• ./logs/condor_logs

Would like to create a setup script that automatically does the following with verbose output describing whats happening:
<env_name> should be a prompt for user input

conda env create -n <env_name> -f conda_requirements.yml
conda activate <env_name>
conda pack -n <env_name> -o nwtenv.tgz
mkdir mongodb/db
mkdir logs
mkdir logs/condor_logs; mkdir logs/errors; mkdir logs/outputs

named: setup.sh

Currently the timeout functionality isn't working and is actually actively ruining runs, possible relation to issue #4 with reading in the run.sh file, possibly not

Flag in NWTOPT.py that will initially automatically convert windows files paths in project folder to linux file paths:

• .\ -> ``
• \ -> /

Probably elegant solution with os.path module, issue dependent on #21 to be closed first

Tired of going through project files and manually Ctrl-f finding replacing... :(

need to remove the wrapt_timeout_decorator from the requirements for the python environment since the solution to issue #6, removes the need for it

Convert following code to use the new HParam object for future MF6 compatibility

Current code has almost no comments and need to be added

I tried starting NWTOPT using the command lines you sent. Seems like things start up but then hang.
I'm getting an error in the logfile:

INFO (hyperopt.mongoexp): job exception: [Errno 2] No such file or directory: 'run.sh'

Do I need to copy run.sh to my project directory? I have it in the main NWTOPT directory.

Also, I changed the port so I wouldn't clash with other runs (to 27020) in the command lines and nwtopt.sub.

I didn't run nwtopt.sh manually anywhere, just

mongodb/bin/mongod --dbpath mongodb/db --bind_ip 130.11.177.99 --port 27020
 
condor_submit nwtopt.sub
 
cd NWT_SUBMIT/NWTOPT_FILES; python optimize_NWT.py --ip 130.11.177.99 --port 27020 --key badger_19F --trials 3000

Am I missing something?

current documentation does not result in the proper installation of the conda environment

Would love to have an extension to NWTOPT for modflow-6. Mike already has the beginnings, but would be great to get this done after getting the main nwtopt stuff done.

The IPRNWT parameter just sets printing of iteration information so it probably should be consistently set to 0 or 1.

Setting it to 0 gives a slightly faster overall time because MODFLOW-NWT doesn't have to write out all
the iteration information; but its nice to have it set to 1 for checking model performance. Maybe the time difference between the same set of parameters with IPRNWT 0 or 1 is not that important; but it should be something.

I suggest setting it to 0 for NWTOPT, users can easily flip it to 1 if they want to look at where the model
is getting bogged down. You are not saving the list files in NWTOPT, so setting the flag to 1 would not
be helpful in the optimization.

HWR

Purge historic ips from git history, then recreate file, then create precommit hook that will change .sub file to be generic

Simple fix, more professional, should be done soon

This part of the objective function is redudant:

I remember @hwreeves-USGS, @rjhunt-usgs, and I had a conversation on a different loss function that would incorporate MB throughout each step, but I'm forgetting what the suggestion for change was. Would either of you be able to clarify what that idea was? It was something like an averaging of per-step MB I think...

Looks like the example run.sh script says to enter timeout value in minutes but the code is converting to seconds.....just need to choose :)

Max,
Looking at objective.py, after it finds mass_balance around line 85, should it be made strictly positive?

mass_balance = np.abs(mass_balance)?

I guess you can't do it in the try statement because it will throw an error for characters in the line.

It must not be a big deal or you'd be getting strange results, but just in case it seems like making sure the mass balance error used in the loss function is positive would be a good idea.

HWR

So keys used for runs don't have to be remembered and to index the database.

Currently the codebase is messy, confusing, uses globals, and overall a little gross. Planning on creating an object orientated approach in the solution to this issue.

documentation leaves out file structure information and non-condor run execution instructions. These should be added before a "formal" release of the software.

The user needs to add mongodb/db and logs/condor_logs, logs/outputs, and logs/errors

NWTOPT worked after adding these - fixing the earlier issues.