Flow differences between S09I05 (CALVIN) and pyvin about calvin

[jdherman]

Thanks a lot Mustafa. Really excited that we're getting results now!

Some of these are very different -- but they seem to be in places that are similar to the differences we saw in Hobbes vs. PRM comparison. Could it be the case that the differences are in the network data, not the solver?

Another point is that our cost curve extensions are different than PRM, so we shouldn't expect exactly the same answer.

How do the objective function values compare?

[jdherman]

Just for comparison here is the portfolio plot from Max's script:

(y-axis in units of average TAF/month)

compared to these figures from Dogan (2015)

Overall it matches up pretty well! (our new plot doesn't show shortage, and doesn't include the warm-dry scenario).

[msdogan]

Yes, looks like flow differences are occurring on links that we saw some differences while comparing CALVIN and HOBBES, but this time differences are even bigger. It is possible that the issue is input data. We need to go back check c, a, lb, ub parameters for these links to see if there is any problem.

For the objective function, values are close, at least in the same order of magnitude:
S09I05 z = -40664453805.7
pyvin z = -40676283943.10019
difference = 11830137
So, since the objective is to minimize cost, CPLEX seems to do better job if we represent everything correctly.

[msdogan]

Thanks for the plots. I am guessing values on Max's plot are TAF/y instead of TAF/m. And looks like it is about 100 to 200 TAF/y less than what I calculated. The difference is scarcity?

[jdherman]

Hey they both get $40.6 Billion for the optimal cost -- that's pretty close! I feel good about that.

For the flow volumes I agree we can double-check the c,a,l,u parameters.

I'm pretty sure the bar plot is TAF/m. If you multiply by 12 and divide by 1000 (so basically divide by 100) you get values that are close to yours. And yes scarcity is not included, you're right.

[msdogan]

Sorry, yeah, ag and urban deliveries are looking close.

[msdogan]

Here is the latest comparison of CALVIN and PyVIN (run 12272016) after Justin's updates on MIF representation. PyVIN was run in debug mode but total debug flow amount is small #29. I compared some examples from network matrix (input) to CALVIN, and there is no issue (#35). Everything (bound & cost) seems to be correctly represented in network matrix, however there might be some local issues.

Classical cdf :)

links that have monthly average flow difference > 10 TAF/m

Network Link	S09I05	pyvin	S09I05-pyvin (difference)
D513-D521	32.70	55.85	-23.15
C83-HSU202C83	2.88	25.07	-22.19
D521-D522	291.99	313.26	-21.27
HSU202C83-A202	2.54	22.06	-19.53
D800-D801	327.79	338.69	-10.90
D801-D803	327.65	338.55	-10.90
D804-D814	326.94	337.83	-10.89
D803-D804	327.58	338.47	-10.89

C83-C31 	151.13	128.93	22.20
C31-D37 	482.03	459.82	22.22
HSU202C80-A202	58.14	35.62	22.51
D37-C32 	507.47	483.43	24.04
C80-HSU202C80	66.60	41.02	25.58
C32-C80 	66.57	40.99	25.58
D515-D522	528.01	501.13	26.88
D513-D515	486.53	458.96	27.57

[jdherman]

Oh much better!! Last time the biggest differences were 100-200 TAF/m on average, now they're 10x smaller. We'll run it by the group but I think this is close to done: the objective function is the same, debug flows are very small, so maybe these differences are just multiple optima.

[msdogan]

Yes, differences are much smaller in this run. We can also compare reservoir storage/release, ag and urban deliveries, where we apply economics, and then decide how to proceed.

[jdherman]

Ok that's great -- Let's do those comparisons after the break. @mfefer this would be a good application of your postprocessing scripts :)

[msdogan]

Objective value comparison of latest 82-year run (https://github.com/msdogan/pyvin/tree/master/examples/statewide_82-year_results)

model	Objective Value ($)
S09I05	-40,664,453,806
pyvin	-40,668,750,069
Difference ($)	-4,296,264
Difference (%)	0.011