pyomo / pyomo

An object-oriented algebraic modeling language in Python for structured optimization problems.

Pyomo Overview

Pyomo is a Python-based open-source software package that supports a diverse set of optimization capabilities for formulating and analyzing optimization models. Pyomo can be used to define symbolic problems, create concrete problem instances, and solve these instances with standard solvers. Pyomo supports a wide range of problem types, including:

• Linear programming
• Quadratic programming
• Nonlinear programming
• Mixed-integer linear programming
• Mixed-integer quadratic programming
• Mixed-integer nonlinear programming
• Mixed-integer stochastic programming
• Generalized disjunctive programming
• Differential algebraic equations
• Mathematical programming with equilibrium constraints
• Constraint programming

Pyomo supports analysis and scripting within a full-featured programming language. Further, Pyomo has also proven an effective framework for developing high-level optimization and analysis tools. For example, the mpi-sppy package provides generic solvers for stochastic programming. mpi-sppy leverages the fact that Pyomo's modeling objects are embedded within a full-featured high-level programming language, which allows for transparent parallelization of subproblems using Python parallel communication libraries.

• Pyomo Home
• About Pyomo
• Download
• Documentation
• Performance Plots

Pyomo was formerly released as the Coopr software library.

Pyomo is available under the BSD License - see the LICENSE.md file.

Pyomo is currently tested with the following Python implementations:

• CPython: 3.8, 3.9, 3.10, 3.11, 3.12
• PyPy: 3.9

Testing and support policy:

At the time of the first Pyomo release after the end-of-life of a minor Python version, we will remove testing for that Python version.

Installation

PyPI

pip install pyomo

Anaconda

conda install -c conda-forge pyomo

Tutorials and Examples

• Pyomo — Optimization Modeling in Python
• Pyomo Workshop Slides
• Prof. Jeffrey Kantor's Pyomo Cookbook
• The companion notebooks for Hands-On Mathematical Optimization with Python
• Pyomo Gallery

Getting Help

To get help from the Pyomo community ask a question on one of the following:

• Use the #pyomo tag on StackOverflow
• Pyomo Forum

Developers

Pyomo development moved to this repository in June 2016 from Sandia National Laboratories. Developer discussions are hosted by Google Groups.

The Pyomo Development team holds weekly coordination meetings on Tuesdays 12:30 - 14:00 (MT). Please contact [email protected] to request call-in information.

By contributing to this software project, you are agreeing to the following terms and conditions for your contributions:

1. You agree your contributions are submitted under the BSD license.
2. You represent you are authorized to make the contributions and grant the license. If your employer has rights to intellectual property that includes your contributions, you represent that you have received permission to make contributions and grant the required license on behalf of that employer.

Related Packages

See https://pyomo.readthedocs.io/en/latest/related_packages.html.

pyomo's Issues

display() and pprint() not working on SimpleConnector

I realize that a rewrite of the Connector object implementation is in the works. Wanted to bring this issue to your attention as you're doing that.
Error messages:

>>> w.units.out_mix.block.out_port.display()
Connector out_port :  Size=1Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/qichen/.solvers/pyomo/src/pyomo/pyomo/core/base/connector.py", line 302, in display
    ', '.join(sorted(self._conval[None].keys()))+"}"+'\n')
TypeError: sequence item 0: expected string, NoneType found
>>> w.units.out_mix.block.out_port.pprint()
  out_port :    Size=1Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/qichen/.solvers/pyomo/src/pyomo/pyomo/core/base/connector.py", line 277, in pprint
    if self._index_set is not None:
AttributeError: 'SimpleConnector' object has no attribute '_index_set'

The structure of the connector is as follows:

def conc_connector(flow, frac):
    c = Connector()
    c.add(flow, name="F")
    for key in frac._index:
        c.add(frac[key], name="conc_{0}".format(key))
    return c

Solver options for scip using pyomo

I was wondering if there was any way to set solver options such as mipgap for scip. This is a list of all the parameters that scip accepts.

Change in scope causes x = 1 aliased Var value assignment to fail

m = ConcreteModel()
m.x = Var(domain=Binary)
x = m.x
if x is not None:
    x.display()  # works properly
    x = 1
    x.display()  # error, because x is now an int

If I had used x.value = 1, this would not be a problem, but I recall @jsiirola telling me that the recommended usage was direct assignment, i.e. x = 1 since there is talk of value changing to be a property. This would seem to be an unintentional consequence.

Add SMPS conversion to the 'convert' subcommand

Converting SP problems is only supported with a separate subcommand, but it should be integrated into the 'pyomo convert' subcommand.

glpk: invalid literal for int() with base 10: 'c'

Hi,
we have a problem with the glpk solver:

 pyomo solve --solver=glpk diet1.py diet.dat
[    0.00] Setting up Pyomo environment
[    0.00] Applying Pyomo preprocessing actions
[    0.00] Creating model
[    0.07] Applying solver
invalid literal for int() with base 10: 'c'
[    0.08] Pyomo Finished
ERROR: Unexpected exception while running model:
    Error parsing solution data file, line 1
$ exit

Pyomo is 4.3.11388, glpksol is 4.60.

Create a new component: MutableParam

The discussion related to PR #33 highlighted that immutable and mutable parameters have very different semantics. For example, PR #33 allows mutable parameters to be defined and used to create expressions without having values, which immutable parameters would generate an error in this context.

I think it may make sense to have separate components: Param and MutableParam that segregate these two notions of "data values" in models. What are the pros and cons?

Bug in Abstract Set Creation

In previous versions of Pyomo, one could use set expressions on abstract sets (which makes sense). In the current version, and on trunk, the following sort of thing:

model.S_Diff = Set()
model.S_Norm = Set()
model.ASS = model.ASS_pro|model.ASS_cst
model.ASSPRO = model.ASS|model.PRO
model.ASSPRODIS = model.ASSPRO|model.DIS
model.S = model.S_Norm|model.S_Diff

results in something like:

ERROR: Unexpected exception while loading model:
Cannot iterate over abstract Set 'S_Norm' before it has been constructed (initialized).

This code works in previous versions of Pyomo. A example for testing is attached (if the example does not result in the error message "Cannot iterate over abstract Set ..." then it is no longer useful. It is not intended to be a working example).

setbug.zip

Can't add to an index connector until owned by a model or block

I can add stuff to a connector before I assign it to a block, but not to an indexed connector. If this is expected behavior, I can work with it, but I think it may be a bug. See this example

from __future__ import division
from __future__ import print_function

from pyomo.environ import *
from pyomo.core.base.connector import ConnectorExpander

model = ConcreteModel()
model.x = Var(initialize=1, domain=Reals)
model.y = Var(initialize=2, domain=Reals)
c = Connector([1,2])
c[1].add(model.x, name='v')
c[2].add(model.y, name='v')
model.c = c

Get this error:

jeslick@pds04:/base_units> python indexed_connector.py
Traceback (most recent call last):
File "indexed_connector.py", line 11, in
c[1].add(model.x, name='v')
File "/imports/users/jeslick/anaconda2/lib/python2.7/site-packages/pyomo/core/base/connector.py", line 212, in getitem
raise KeyError(msg)
KeyError: "Unknown index '1' in connector IndexedConnector; Make sure the correct index sets were used.\n Is the ordering of the indices correct?"
jeslick@pds04:/base_units>

is_fixed checks inside constraint.set_expr need to be changed

The checks for is_fixed when setting a constraint expression need to be changed to something more restrictive. However, is_constant is too restrictive. We need another expression classifier (is_numeric?) that checks for the existence of optimization variables, or even the possibility of them appearing in the future (e.g., because a mutable expression appears somewhere). Here is some pseudo code of a problematic case:

y = variable()
x = variable()
expr = x == y
x.fix(1.0)
c = constraint(expr)
x.free()

Now c.lower and c.upper are equal to x (which are always supposed to be "numeric"), but there is nothing invalid about how this constraint was defined. It is simply a side-effect of our invalid code breaking apart the constraint expression in a way that doesn't make sense.

pyomo.dae transformations ignore Piecewise blocks

The issue should be explained near the end of the following thread:
https://groups.google.com/forum/#!topic/pyomo-forum/Q_60FKCXqkw

It looks like Piecewise does not get re-constructed over the expanded ContinuousSet .

MIP solver option - Resolve with fixed integer variables

The following link illustrates an issue with MIP solvers: they may not return feasible solutions for their continuous variables:

http://yetanothermathprogrammingconsultant.blogspot.com/2016/08/cplex-bug-or-not.html

We should consider adding an option in Pyomo to force MIP solvers to resolve with their fixed integer variables.

_ConnectorValue Missing Attributes

Still having trouble with indexed connectors these attributes seem to be missing: 'value' and 'component'. Not sure how these things work, but also missing from slots like name and domain were? I added them there and go a bit further.

Derived component IndexConstraint failed to define _default()

Got the following error trying to refer to a Constraint that I skipped definition for.

File "/home/qtothec/pyomo/src/pyomo/pyomo/core/base/indexed_component.py", line 516, in _default
  % (self.__class__.__name__,))
NotImplementedError: Derived component IndexedConstraint failed to define _default().
    Please report this to the Pyomo developers

It's quite easy to produce this error:

from pyomo.environ import *
m = ConcreteModel()
m.x = Var()
def _c(b, i):
    if i == 1:
        return m.x == 1
    else:
        return Constraint.NoConstraint
m.c = Constraint([1, 2, 3], rule=_c)
print(m.c[2])
# or really anything else trying to refer to that _ConstraintData
# is going to give the above error

I'm not opposed to getting an error message in this case, since it was an error on my part, but the message should probably say something different.

Related to this, if I'm iterating over m.c._index, would be best way to check for existence be to use indx in m.c?

IndexedVar domain display not updated

When the domain of a Var is not preset and its elements are modified, the default domain of Reals is displayed even when all elements are changed consistently to a different domain. Small example below:

"""
Small example showing outdated Var domain display
"""

from pyomo.environ import ConcreteModel, Var, RangeSet, Binary

m = ConcreteModel()
m.s = RangeSet(2)
m.x = Var(m.s)
m.x[1].domain = Binary
m.x[2].domain = Binary

m.x.display()

"""Result is
x : Size=2, Index=s, Domain=Reals
    Key : Lower : Value : Upper : Fixed : Stale
      1 :     0 :  None :     1 : False :  True
      2 :     0 :  None :     1 : False :  True
despite both individual vars having a domain of Binary
"""

Recursion error when constructing Var indexed on [None]

I suspect that something to do with process_setargs is causing a recursion error in the following example:

m = ConcreteModel()
m.x = Var([None])

In case this seems like a silly thing to do, I encounter it when I am trying the following:

m = ConcreteModel()
m.x = Var()

m2 = ConcreteModel()
m2.x = Var(m.x._index)

In this case, I could just use clone instead, but if I am just trying to clone certain variables, that would be overkill.

Trivial Constraints and Empty Rows in NL file

This issue is a follow up from a Pyomo forum discussion.

CONOPT complains about empty rows being sent to it in an .nl file. In a larger model, this may happen to rows corresponding to logic constraints when a subset of discrete variables are fixed. While one remedy would be to deactivate the affected equations, it could be difficult to keep track of them. According to the CONOPT developer, AMPL will filter empty rows out on the way into CONOPT and add the proper row information (status and duals) for the empty rows on the way back. Is this something that we could do as well?

Code files for a small test case are included in a Gist: https://gist.github.com/qtothec/30b5edd277e74171088dbb91b11ea77e

del_component issue when deleting constraints

I'm working on a project that frequently requires solving a model, modifying a constraint, and then resolving. I've been using the del_component & add_component commands for removing and recreating constraints (see simple attached example). It appears that the del_component(constraint of interest) command is leaving behind some garbage if the constraint to be removed is indexed. This necessitates an additional command (i.e., "model.del_component(model.constraint1_index)") to avoid an error.

Is this a bug? Can it be fixed? Should I be doing something differently?

Thanks!
Tony Burgard (NETL)

Simple Example.py.txt

IndexedConnector missing 'aggregators'

When I go to expand an indexed connector, I get an error about missing aggregators. See this example:

from __future__ import division
from __future__ import print_function

from pyomo.environ import *
from pyomo.core.base.connector import ConnectorExpander

model = ConcreteModel()
model.x = Var(initialize=1, domain=Reals)
model.y = Var(initialize=2, domain=Reals)
model.c = Connector([1,2])
model.c[1].add(model.x, name='v')
model.c[2].add(model.y, name='v')

model.eq = Constraint(expr=model.c[1] == model.c[2])
model.o = Objective(expr = model.x**2)

xfrm = ConnectorExpander()
xfrm.apply(instance=model)

opt = SolverFactory('ipopt')
results = opt.solve(model)

model.display()
print(results)
print("x = {}".format(model.x.value))
print("y = {}".format(model.y.value))

Gets this error:

jeslick@pds04:/base_units> python indexed_connector.py
Traceback (most recent call last):
File "indexed_connector.py", line 16, in
xfrm = ConnectorExpander()
NameError: name 'ConnectorExpander' is not defined
jeslick@pds04:/base_units> python indexed_connector.py
Traceback (most recent call last):
File "indexed_connector.py", line 18, in
xfrm.apply(instance=self.pyomo_block)
NameError: name 'self' is not defined
jeslick@pds04:~/base_units> python indexed_connector.py
Traceback (most recent call last):
File "indexed_connector.py", line 18, in
xfrm.apply(instance=model)
File "/imports/users/jeslick/anaconda2/lib/python2.7/site-packages/pyomo/core/base/connector.py", line 467, in apply
for var, aggregator in iteritems(conn.aggregators):
AttributeError: 'IndexedConnector' object has no attribute 'aggregators'

ExternalFunction with fixed variables not working

I am able to use ExternalFunction ok in Pyomo, but the expressions cannot be evaluated in Python. If the arguments to the function happen to be fixed, when the problem is sent to the solver, I think Pyomo tries to evaluate the function with fixed arguments and turn it into a constant. This causes an error like ExternalFunction cannot be evaluated by Python.

I'm not able to get the exact error message right now, but I can add details (even a sample problem) if needed.

Incorrect behavior with the relax_integrality solver option

The way this option is handled in the CPLEX solver plugin is to simply change the problem type to an LP. However, this produces incorrect behavior when the problem has any quadratic constraints or objective terms (they get ignored).

Also, this is technically not a solver option. It should probably be managed using a keyword to the solve command.

Resolve hanging tests: Pyomo_trunk_parallel_py35

This test is hanging because it is not connecting with CPLEX. Gabe has some fixes, so I'm assigning it to him.

Deprecate the Pyomo functor API

As far as I can tell, none of our core developers are using this. We should do an informal poll, and confirm whether this API can be deprecated.

Fix confusing error message in bilevel transform for nonlinear models

This transform implicitly assumes it is given a linear model by treating the constraint canonical repn as a LinearCanonicalRepn object. This leads to a confusing error message when the model has nonlinear constraints.

Indexed connector not working

I cannot get indexed connectors to work. Here is an example, where the last line fails.

from __future__ import division
from __future__ import print_function

from pyomo.environ import *

model = ConcreteModel()
model.c1 = Connector()
model.c2 = Connector([1,2])

The error is:

python indexed_connector.py
Traceback (most recent call last):
File "indexed_connector.py", line 8, in
model.c2 = Connector([1,2])
File "/imports/users/jeslick/anaconda2/lib/python2.7/site-packages/pyomo/core/base/connector.py", line 371, in new
self = IndexedConnector(_args, *_kwds)
File "/imports/users/jeslick/anaconda2/lib/python2.7/site-packages/pyomo/core/base/connector.py", line 331, in init
self._dummy_val = _ConnectorValue(kwds.get('name', None))
File "/imports/users/jeslick/anaconda2/lib/python2.7/site-packages/pyomo/core/base/connector.py", line 48, in init
self._name = name
AttributeError: '_ConnectorValue' object has no attribute '_name'

"Integral" is calling the base class constructor incorrectly

The question I asked on stackoverflow illustrates the confusion around defining an integral: http://stackoverflow.com/questions/39908555/how-to-define-an-integral-as-an-objective-function-in-pyomo
It seems like it fails the type-checking, even though it shouldn't.

Integrate PySP commands into the 'pyomo' command

A while back we integrated most of Pyomo's commands as subcommands of the pyomo command. The exception was PySP, which was in flux at the time. I'm not sure we can do this before the 5.0 release, but this still should be done.

I'm open to suggestions for the subcommands that are used, particularly for Pyro.

Bug using numpy float values in Pyomo expressions.

Consider the following script:

# pyomo 4.4.1
# python 3.5
from pyomo.environ import *
import numpy

m = ConcreteModel()

m.T = RangeSet(10)
m.v = Var(initialize=1, bounds=(0,None))
m.c = Var(m.T, initialize=20)

def rule(m, t):
    h = numpy.float32(1.0)
    # This works
    return m.c[t]  == h * m.c[t]
m.X = Constraint(m.T, rule=rule)

def rule(m, t):
    h = numpy.float32(1.0)
    # This works
    return m.c[t]  == h * m.c[t] * m.v
m.X = Constraint(m.T, rule=rule)

def rule(m, t):
    h = numpy.float32(1.0)
    # This fails
    return m.c[t]  == h * m.v
m.x = Constraint(m.T, rule=rule)

The first two constraints are constructed fine, but the last one fails, generating the following error:

ERROR: Rule failed when generating expression for constraint x with index 1:
    KeyError: "Error accessing indexed component: Cannot treat the scalar component 'v' as an array"
ERROR: Constructing component 'x' from data=None failed:
    KeyError: "Error accessing indexed component: Cannot treat the scalar component 'v' as an array"
Traceback (most recent call last):
  File "tmp.py", line 26, in <module>
    m.x = Constraint(m.T, rule=rule)
  File "/Users/wehart/home/src/python35/src/pyomo/pyomo/core/base/block.py", line 483, in __setattr__
    self.add_component(name, val)
  File "/Users/wehart/home/src/python35/src/pyomo/pyomo/core/base/block.py", line 849, in add_component
    val.construct(data)
  File "/Users/wehart/home/src/python35/src/pyomo/pyomo/core/base/constraint.py", line 736, in construct
    ndx)
  File "/Users/wehart/home/src/python35/src/pyomo/pyomo/core/base/misc.py", line 59, in apply_indexed_rule
    return rule(model, index)
  File "tmp.py", line 25, in rule
    return m.c[t]  == h * m.v
  File "/Users/wehart/home/src/python35/src/pyomo/pyomo/core/base/indexed_component.py", line 499, in __getitem__
    raise KeyError(msg)
KeyError: "Error accessing indexed component: Cannot treat the scalar component 'v' as an array"

DAE Toolbox cannot discretize indexed Expression objects

There is an issue with discretizing indexed expression objects at the moment. You get the error:

AttributeError: 'IndexedExpression' object has no attribute '_add_members'

This is probably due to a change of attribute names from '_add_members' to 'add' in the Expression object syntax. You can reproduce this error with this short example:

from pyomo.environ import *
from pyomo.dae import *

m = ConcreteModel()
m.z = ContinuousSet(bounds=(0,10))

def _test(m,z):
  return 1

m.test = Expression(m.z, rule=_test)

disc = TransformationFactory('dae.finite_difference')
disc.apply_to(m, wrt=m.z, nfe= 10,  scheme= 'BACKWARD')

A simple fix of this issue would be reformulating the '_update_expression' function in the 'misc' module of the dae toolbox to something like this:

def _update_expression(expre): 
   """ This method will construct any additional indices in a expression 
   resulting from the discretization """

    for i in expre.index_set():
      if i not in expre:        
        _rule=expre._init_rule
        _parent=expre._parent()

        expre.add(i, apply_indexed_rule(expre,_rule,_parent,i))

In addition, the 'construct' attribute of the Expression object needs to be altered so that the '_init_expr' and '_init_rule' are no longer deleted after member initialization.

Error using uninitialized mutable Param as constraint bound

When I do something like the following:

from pyomo.core import *
model = ConcreteModel()
model.x = Var()
model.p = Param(mutable=True)
model.c = Constraint(expr= model.x <= model.p)

I immediately run into the following error message:

ValueError: Constraint 'c' created with a -Inf upper bound.

The error message goes away if I initialize the mutable Param to some finite value.

I don't think this kind of logic really belongs here. It would also be very easy to change the mutable Param value to "infinity" later on anyway. This case probably needs to be handled by the solver plugins at run time by either skipping unbounded constraints, throwing an error, or passing the constraint on to the solver as is (which some input formats support).

Appveyor tests are always "passing"

The root cause may be the return value for test.pyomo.exe. I suspect that it's not returning a nonzero value when a test fails.

Create a default model name based on the file name

Currently, the default model name is 'unknown'. I think it makes sense to create a default model name based on the Python file, at least when running the 'pyomo' command.

For example, the command:

pyomo convert --format=lp foo.py

generates the file 'unknown.lp' unless the user defines a model name. A more sensible default would be to label the model with the name 'foo'.

ExternalFunction getname() not working

I'm trying to use the symbolic_solver_labels = True option with a solver, so I can relate constraints in the nl file to constraints in Pyomo. There seems to be a problem with the getname() function in external.py that prevents it from working right.

I looks like getname() in external.py calls getname() in component.py which calls the getname() function in external,... until the recursion depth limit. If I just stick a return in getname() in external.py, I can get it to work fine, although it wouldn't return the right name.

I put an example here: https://github.com/eslickj/bug_examples/blob/master/fixed_vars_in_external

AttributeError when using glpk with mipgap option

Hi,

After passing the mipgap option to glpk (and glpk terminating with message "RELATIVE MIP GAP TOLERANCE REACHED; SEARCH TERMINATED"), the following error is produced:

File "/Users/username/anaconda/lib/python3.5/site-packages/pyomo/solvers/plugins/solvers/GLPK.py", line 481, in _process_soln_mip
solv.termination_condition = TerminationCondition.feasible
AttributeError: 'Enum' object has no attribute 'feasible'

Is this a bug? Can you suggest a solution?

Thanks,
Jason

XPRESS Solver Error : AttributeError: 'NoneType' object has no attribute 'problem'

I've pasted the traceback below :

Traceback (most recent call last):
  File "model.py", line 15, in <module>
    solver.solve(model)
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/opt/base/solvers.py", line 587, in solve
    result = self._postsolve()
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/opt/solver/shellcmd.py", line 267, in _postsolve
    results = self.process_output(self._rc)
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/opt/solver/shellcmd.py", line 329, in process_output
    self.process_soln_file(results)
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/solvers/plugins/solvers/XPRESS.py", line 336, in process_soln_file
    results.problem.number_of_objectives=1
AttributeError: 'NoneType' object has no attribute 'problem'

stdout and stderr ordering issues when streaming solver output

I believe that the order of stdout and stderr is not preserved when streaming solver output.

Details:
I have attached fm7.py. During solution of this problem, IPOPT (or rather ASL) encounters an evaluation error (sqrt of a negative number). The option "halt_on_ampl_error" is set to "yes", therefore there is an error printed to the console that indicates the problematic expression/constraint (from an ASL standpoint).

However, this error does not appear at the end of the console output (when it actually occurred), but rather up in the middle of the output, leading to confusion of where this error occurred in the solution process.

This can be reproduced if IPOPT is installed (with MUMPS) using the following command:
$ python fm7.py
My output from this command is stored in fm7.pyomo.out.

If we run the ".nl" file manually with IPOPT, we see the correct ordering. The details of the error are printed at the end (along with the correct iteration where the error appeared). This can be reproduced with the following command:
$ ipopt path/to/fm7.nl halt_on_ampl_error=yes
My output from this command is stored in fm7.ipopt_nl.out

While ordering of stdout and stderr to the screen may not be guaranteed, I suspect that IPOPT manually flushes the stdout buffer to ensure the correct order. This is lost in Pyomo.

ticket.zip

Automatic publishing from Jenkins is broken

After the move from the old to the new Jenkins server, automatic publication of the following resources to software.sandia.gov is broken:

• pyomo_install
• Pyomo (trunk) asciidoc documentation

Revisit whether 'six' is still necessary

According to this blog, we should be able to forgo the use of the 'six' package now that we're not supporting Python 2.5. I'm not sure I believe that, but it's probably worth considering this.

http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/

XPRESS.py always uses lpsolve irrespective of problem type

The XPRESS.py plugin using lpsolve irrespective of the problem type. This results in an incorrect solution when solving MILP problems. The correct solver is mipoptimize for MILP problems. XPRESS.py should either

1. Ask the user for the problem type
2. Have pyomo figure out the problem type

XPRESS: NameError: global name 'field_name' is not defined

XPRESS.py does not define field_name before using it. The following is the traceback from running the example in #58

$ python test.py
Traceback (most recent call last):
  File "test.py", line 14, in <module>
    results = solver.solve(m)
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/opt/base/solvers.py", line 587, in solve
    result = self._postsolve()
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/opt/solver/shellcmd.py", line 267, in _postsolve
    results = self.process_output(self._rc)
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/opt/solver/shellcmd.py", line 329, in process_output
    self.process_soln_file(results)
  File "/Users/$USER/anaconda/lib/python2.7/site-packages/pyomo/solvers/plugins/solvers/XPRESS.py", line 354, in process_soln_file
    if (extract_reduced_costs is True) and (field_name == "reducedCost"):
NameError: global name 'field_name' is not defined

Document all Pyomo transformations

Several of the DAE transformations have "TODO" in their transformation documentation.

Make modeling checking the default behavior for the 'pyomo' command

I'm adding this issue to force a decision about whether checking is the default behavior. Given recent discussions regarding categories of Pyomo users, I think it makes the most sense for checking to be the default behavior. This is a minor code change, but we should advertise it in advance of the 5.0 release if we do this.

With cplex, solver_io='python' is much slower than solver_io='nl' or default

A comment on stackoverflow mentioned that CPLEX's python bindings should be faster than the 'nl' (AMPL-style) interface or the default (which I think interacts with the cplex executable directly), especially with Pyomo versions >= 4.3.11323. However, on my system I have not found this to be the case. In the example below, the solver takes about 8x longer when using the solver_io='python' option than it does with the other options. This is on Mac OS X 10.11.6, with Python 2.7, CPLEX 12.6.0.0 and Pyomo 4.4.1.

import time
from pyomo.environ import *
from pyomo.opt import SolverFactory

def solve(*args, **kwargs):
    # define the model
    n_steps = 100000
    m = ConcreteModel()
    m.STEPS = Set(initialize=range(n_steps))
    m.target = Param(m.STEPS, initialize=lambda m, s: 1.0)
    m.Build = Var()
    m.Operate = Var(m.STEPS)
    m.Meet_Target = Constraint(m.STEPS, rule=lambda m, s: m.Operate[s] >= m.target[s])
    m.Operate_Limit = Constraint(m.STEPS, rule=lambda m, s: m.Operate[s] <= m.Build)
    m.MinCost = Objective(
        rule=lambda m: n_steps * m.Build + sum(1.0 * m.Operate[s] for s in m.STEPS), 
        sense=minimize
    )

    # solve the model
    opt = SolverFactory(*args, **kwargs)
    start_time = time.time()
    results = opt.solve(m)
    m.solutions.load_from(results)

    print "solve({}): MinCost={}, time={}".format(
        ", ".join([repr(a) for a in args] + [str(k)+"="+repr(a) for k, a in kwargs.items()]),
        value(m.MinCost), time.time() - start_time)

solve("cplex")                      # 9.2s
solve("cplex", solver_io="nl")      # 9.2s
solve("cplex", solver_io="python")  # 73.3s
solve("cplexamp")                   # 9.2s

I am pleased to note that these times are all 20-35% faster in Pyomo 4.4.1 than they were in 4.3.11377. But the "python" bindings are clearly lagging.

Export coverage from Jenkins jobs

We should augment the Jenkins tests to also push their coverage results to the codecov website. I have set up the codecov client application, so (hopefully) the only thing left to do is for @whart222 to generate whatever tokens we need to include in the call to codecov to get the results pushed / merged into the master report. See the codecov documentation.

Other issues:

• Our current Jenkins tests only test master and not pull requests. Should we look at adding that?
• Will the coverage information for PR be skewed because the PR coverage won't include the tests that require solvers (and are only run in Jenkins)?

Baron does not work with the Pyro solver manager

Reported on the pyomo forum by @qtothec:
https://groups.google.com/forum/#!topic/pyomo-forum/EmVkqIjJYXg

Pyomo should be able to load external functions from .so file in LD_LIBRARY_PATH

Currently pyomo only loads external function property definitions from a phys_prop.so file in the current directory. It should search LD_LIBRARY_PATH (or equivalent) location for this file.

Python 3 compatability of external.py

I know this is pre-alpha functionality and not seeing much development, but I have been trying to use ExternalFunction calls using Python 3, and have run into compatibility issues.

The first issue that appeared was due to the difference in behaviour of the range function between Python 2 and 3 (lines 57 and 58 of external.py). I tried fixing this by changing line 57 from:

idxs = range(len(args))

to:

idxs = list(range(len(args)))

which resolved that issue, however this led to other issues. In my particular case, this appears to be in line 104 (indicating an issue with my external function not being registered), where the code:

', '.join(self._known_functions.keys()) ) )

returns the error: TypeError: sequence item 0: expected str instance, bytes found

If you could look into updating external.py to support Python 3, it would be much appreciated.

Regards,
Andrew Lee

Unable to pass solver options to neos

Hi,

I'm using the neos solver manager with the cplex solver, but I appear to be unable to pass solver options.

Note that I can pass options to solvers on my local machine.

Is this a bug? Do you have any suggestions?

Thanks,
Jason

Using constant Params as an index

In the following code snippet, building the objective function fails because the get item function on an indexed component does not automatically replace a constant Param with its value.

m.tf = Param(initialize=1)
m.t = ContinuousSet(bounds=(0,m.tf))
m.x = Var(m.t)

m.obj = Objective(expr=m.x[m.tf])

We could of course take the value of the parameter explicitly in the objective expression but this isn't as elegant and I think most users would expect the above code to work.

unsupported operand type(s) for *: '_IndexedSetData' and 'SimpleSet'

Hi,

John told me it's probably a bug and I should open an issue here.

in the model

model.Dummy = Set(model.Jobs, within=model.Jobs)
model.Cars = Set()

in the dat-file

...
set Dummy[A_E] := A_E B_E;
...
set Cars := Car1 Car2 Car3;

Now the following constraint wouldn't work:

def JobOnlyOnce_rule(model,j):
    return sum(model.x[i,k] for (i,k) in model.Dummy[j]*model.Cars) <= 1
model.JobOnlyOnce = Constraint(model.Jobs, rule=JobOnlyOnce_rule)

pyomo solve --log bug

Hi,

When I run, for example, pyomo solve --log --solver=glpk diet1.py diet.dat the following error is produced:

ERROR: Unexpected exception while running model:
'str' object has no attribute '_log_file'

Otherwise, the --log option appears to be working. This message is generated on two different machines with different OS, on different problems and solvers.

Is this a bug?

Thanks,
Jason