Comments (5)
Hey @Paszka1 - that is, indeed, a large matrix. I believe pestpp-glm
should be able to handle it.
In any case, using the stability criteria in pestpp-glm
is a good route to follow since based on current parameter values, an appropriate number of single vectors/super parameters can differ. So, I would give it a try and confirm pestpp-glm
can handle it. If you are able to do it that way, the reporting of number of singular values used may be helpful after the fact.
However, if you want to calculate the SVD at the start, I would think you would want to operate on pyemu.la.LinearAnalysis.xtqx
which has square dimensions as number of parameters and 782 should be well within memory.
from pyemu.
Hey Mike,
Thanks for your reply and suggestion to use the xtqx
approach, which worked. I don't perfectly understand why does it need less memory than the qhalf
approach, though.
I have yet another question not directly related to the above. I have generated an ensemble of parameters from a prior. I have one group of grid type parameters for which I haven't specified a geostructure during parameterization, because spatial correlation doesn't really make sense (i.e., they are independent pumping rates), and I haven't specified temporal correlation either. As a result, each realization in the ensemble was generated with the initial value of that parameter, which is of course logical. Is it still possible to draw an ensemble that considers variance for that parameter as well, or shall I specify a structure during parameterization?
from pyemu.
Hi @Paszka1 -
on point 1, cool! Glad that worked. The reason it did is that multiplying XT x Q x X reduces the dimensionality of the matrix from being NOBS x NPAR to NPAR x NPAR. And anyway, it's really the singular vectors of xtqx
that you are interested in.
on the second question, if you are generating your ensemble in PESTPP, it will use the bounds to approximate a variance around the initial parameter value for all the grid parameters. Should be similar in pyemu if you don't specify covariance. So, if they are showing up all at the starting value, maybe you have them set as fixed
in the PST file?
In any case, when you intially sample the ensemble (whether externally through pyemu or your own code, or through PESTPP) that's the only time the covariance/geostructure will play a role, so it's worth making that robust at that point
from pyemu.
Hi @mnfienen,
Should've thought of that (ref. point one).
Point two. Indeed, I fixed those parameters and simply forgot.
I greatly appreciate your replies. They were very helpful notwithstanding that some extra reading and thinking on my end would've probably helped me figure out the answers.
from pyemu.
No worries @Paszka1 - it's alot to take in!
I'm closing this issue. Cheers
from pyemu.
Related Issues (20)
- JCO2JCO functionality in pyemu HOT 3
- Restarting pestpp-glm from pyemu HOT 4
- File distribution functionality in pestpp-glm HOT 2
- read_pestpp_runstorage HOT 5
- SVD-assist if model runs fail during Jacobian calculation HOT 2
- Debugging model infeasibilty after running PESTPP-OPT HOT 2
- pyemu.Pst.from_io_files changed some observation values HOT 2
- ParameterEnsemble from_mixed_draws throws error when there are fixed params and fill=True HOT 3
- Attribute error in apply_genericlist_pars HOT 2
- Calculating phi on ObservationEnsemble HOT 7
- line_terminator now lineterminator HOT 2
- format of parameter data section HOT 3
- Update modflow input files with calibrated values. HOT 7
- Pandas pinned to < 2.1.0 HOT 3
- Out of memory error using Matrix.get with many observations HOT 3
- modelgrid object will not work in place of spatial_reference object in calc_factors_grid HOT 3
- add_parameters KeyError when 'filename' includes path HOT 2
- Justification for setting par_names to all lowercase in parse_tpl_file HOT 2
- use robocopy mirror instead of shutil.copytree in start_workers() HOT 1
- pstfrom_mf6.ipynb doesn't run to completion and returns an error HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyemu.