mitodl / mitx-grading-library

The MITx Grading Library, a python grading library for edX

MITx Grading Library

A library of graders for edX Custom Response problems.

Version 2.4.1 (changelog)

Copyright 2017-2022 Jolyon Bloomfield and Chris Chudzicki

Licensed under the BSD-3 License.

We thank the MIT Office of Open Learning for their support.

Table of Contents

• Demo Course
• Grader Documentation
• Local Installation
• FAQ

Demo Course

A demonstration course for the MITx Grading Library can be viewed here. The source code for this course is contained in this repository here.

Documentation for edX Course Authors

Extensive documentation has been compiled for the configuration of the different graders in the library.

Local Installation

This is not required but can be useful for testing configurations in python, rather than in edX.

To install:

Requirements: An installation of Python 2.7 (for older edX versions) or Python 3.6 or 3.7 (current edX versions).

0. (Optional) Create and activate a new python virtual environment.
1. Clone this repository and cd into it.
2. Run pip install -r requirements.txt to install the requirements specified in requirements.txt. If using python 3, use requirements3.txt instead.
3. Run pytest to check that tests are passing. (To invoke tests of just the documentation, you can run the following command: python -m pytest --no-cov --disable-warnings docs/*)

FAQ

• What's this voluptuous thing?

Voluptuous is a library that handles configuration validation, while giving (hopefully) meaningful error messages. We use it to automate the checking of the configurations passed into the mitxgraders library. They need to be packaged together in the python_lib.zip file.

mitx-grading-library's Issues

Functions to implement in asciimath javascript

These are the new functions that we've introduced that don't have a corresponding function in AsciiMath:

• trace: needs AsciiMath definition (done in #111)
• trans, ctrans, adj, cross: needs preprocessing
• conj: needs an option to write as (...)^*
• Vectors need an option to display as column vectors

Tensor Products

MatrixGrader does not currently support tensor product (aka Kronecker product). This would certain be very very useful for a quantum information course.

@jolyonb Do any of the existing/planned QM series need this?

Checklist for public release

• Create examples in edX example course
• Implement travis/coveralls, add badges!
• Advertise!

Unit Vectors

We should expose cartesianxyz and cartesianijk to make it easy for people to use the hatx etc convention of their choice.

SpecifyDomain and row vs col vs vec

If the domain is specified as 3-component vectors, then matrices of shapes (3, 1) and (1, 3)` will be rejected.

Is this the behavior that we want?

Reduce unncessary nodes in parse tree.

The parser in calc.py attaches sum/product/parallel/power to every set of parentheses, even if there's nothing to add/multiply/etc. For example, ((2)) parses as:

<sum>
  <product>
    <parallel>
      <negation>
        <power>
          <atom>
            <parentheses>
              <sum>
                <product>
                  <parallel>
                    <negation>
                      <power>
                        <atom>
                          <parentheses>
                            <sum>
                              <product>
                                <parallel>
                                  <negation>
                                    <power>
                                      <atom>
                                        <number>
                                          <ITEM>2</ITEM>
                                        </number>
                                      </atom>
                                    </power>
                                  </negation>
                                </parallel>
                              </product>
                            </sum>
                          </parentheses>
                        </atom>
                      </power>
                    </negation>
                  </parallel>
                </product>
              </sum>
            </parentheses>
          </atom>
        </power>
      </negation>
    </parallel>
  </product>
</sum>

it would be nicer (more readable during debugging) we could through away grouped nodes with only 1 child.

Expose multiple comparers

Now that we have custom comparers, I think we should expose a module containing them. Maybe a file called comparers.py in the helpers folder that is just imported by __init__.py as import helpers.comparers. The default comparer (equality) can go in there. Another one would be "between" (does the given expression evaluate between two numbers, raising an error for complex?). Once we have matrix stuff implemented, we'll likely have more ideas.

Correlated Grading

When we have a ListGrader that contains multiple inputs (lowest-level of nesting), the grading for an input should be able to depend on the content of another input. A simple example would be "enter any number here", then "enter its square here". Because the first entry can accept basically anything, that would require a custom comparer. The second entry would also require a custom comparer. The easiest implementation of this would be to allow a custom comparer to have a special parameter that evaluates another input. We'd just need to be careful so as to have no circular references!

Functions and IdentityMultiple

The Issue

In MatrixGrader problems, if IdentityMultiple instances are passed to functions, students currently receive unhelpful generic error messages:

from mitxgraders import FormulaGrader

grader = MatrixGrader(answers='1')
grader(None, 'sin(I)'
# raises:
# StudentFacingError: Invalid Input: Could not check input 'sin(I)'

This is bad.

(The actual cause of this error is an isinf check in calc.py's eval_function parse action. However: I suggest that this issue should be for discussing the intended behavior, not the source of the bug.)

Original motivation of IdentityMultiple

My thinking with IdentityMultiple originally was:

• in MatrixGrader problems, 'I' for identity should always be available, so authors should not need to declare it as a specified constant.
• The dimension of the identity operator is usually irrelevant:
  • True for binary operations add/subtract and multiple/divide
  • True (but not implemented) for functions determinant, transpose, conjugation
  • False for tensor product; False for functions trace, norm
• In specific problems where identity's dimension does matter, the author should declare auser_constant with identity of relevant size. (Either overriding 'I', or declaring 'I_2', 'I_3', etc.)

Resolutions

Shape is a bit more important for the identity matrix than I originally realized. In particular, I see no way to handle trace and norm for generic-dimensional identity operators.

I'm not sure what the best approach to resolve this is. Two ideas:

• Idea A: Implement functions like det, trans, etc, and provide special error messages fortrace andnorm.

  • Pros: Automatically declared I of generic dimension that works in almost all situations.
  • Cons: Hard to maintain lots of special cases. Also, authors need to understand how IdentityMultiple works in order to implement their own matrix functions.

• Idea B: Abandon IdentityMultiple all together. Provide a helper function Identity(n) for authors to declare identity matrix of specific size through user_constants.

  • Pros: More explicit; fewer special cases; automatically resolves #101; identities declared this way would work automatically with tensor products and any numpy matrix function.
  • Cons: Requires extra declaration by author. Requires author to think about dimensionality in problems where it might be irrelevant.

I am personally rather fond of IdentityMultiple, but I am actually leaning toward Idea B.

Matrix/Vector Sampling

We would like to be able to say that A is a matrix of a given size, and sample appropriately. Alternatively, vecv is a vector of a certain size. Students can then input vector math expressions.

• Error should be raised when objects don't line up appropriately.
• We should decide how multiplication works. Is a vector times a vector a dot product?
• The identity matrix will need to be treated carefully.
• Row vs column vectors will need to be decided upon.
• We will probably need to make a transpose operation available, preferably as A^T. If a transposition operation is available, then vectors can be defined as a row/column vector and all multiplications work appropriately (dot products become v^T * v, for example).
• Special matrix samplers can be constructed (symmetric, antisymmetic, orthogonal, traceless, etc).
• Do we want to define a Hermitian conjugate operation? A^dagger?

Overflow errors not being caught

Apparently we have some issues with overflows not being caught in the equality comparer.

from mitxgraders import FormulaGrader

grader = FormulaGrader(
    answers="1",
    debug=True
)

# These two lines raise two separate error messages, neither of which is processed into something useful for students
grader(None, "fact(100)")
grader(None, "fact(200)")

I also suspect that our factorial error messages are never being triggered any more, either.

[Problematic] Vector multiplication is not associative

Since * multiplies vectors as a dot product, * is not associative when used with vectors.

from mitxgraders.helpers.calc import evaluator, MathArray

variables = {
    'i': MathArray([1, 0]),
    'j': MathArray([0, 1]),
}

evaluator("i*i*j", variables)[0]        # result: [0, 1, 0]
evaluator("i*(i*j)", variables)[0]      # result: [0, 0, 0]

This seems like it could cause confusion, especially since edX renders multiplication with dot product symbols.

Edit: In particular, students might not know above whether i*i*j is equivalent to (i*i)*j or i*(i*j).

FormulaGrader Enhancements: Non-scalar formulas

The following are some enhancements to FormulaGrader related to handling non-scalar expressions. Some of these were discussed as summer projects for Chris.

1. Matrix/Vector Sampling: In formulas like a*b, sample a and b from vectors instead of scalars. Useful for grading any non-commutative mathematical expressions. Also possibly useful for specific matrices, like grading Pauli matrix identities in a quantum mechanics course.

   Implementation Difficulty: Straightforward and fast... 1 day for an initial PR?

2. [Implemented in #32] Custom Comparer functions: Currently correctness is determined by comparing student's input and author's answer using approximate equality. Allow author to pass an alternative comparer function. Examples: compare modulo 2π, compare whether expressions are parallel, compare whether student's input is in the span of some other vectors.

   Same as #30.

   Implementation Difficulty: Straightforward and fast... 1 day for an initial PR?

3. [Implemented in #59] Parse/Eval Multivariable: Parse functions with multiple variables, e.g., f(x, y) + g(x)*y^2

   Implementation Difficulty: A bit harder. Minor changes to parser and evaluator

4. [Implemented in #61] Parse/Eval Vectors and Matrices: Parse and evaluate expressions like A*[1,2,3] + b where A is a matrix and b a vector, or [1, x]/sqrt(1 + x^2). Jen has wanted this for linear algebra courses and it could be useful in physics, too. (Although, I suspect that physics prefers Dirac bra-ket notation).

   Implementation Difficulty: Much harder. Significant changes to parser and evaluator

If necessary, we can discuss each of these in-depth in its own issue. Current question:

Question: Should these enhancements all be rolled up into FormulaGrader? Or should the matrix-related enhancements be part of a new grader class?

Strip whitespace from forbidden strings

In FormulaGrader, we strip whitespace from student expression before checking for forbidden strings. We should also strip whitespace from the forbidden strings themselves.

For example,

# Simplify (a + b)^2 - (a - b)^2
FormulaGrader(
    answers='4*a*b',
    variables=['a', 'b']
    forbidden_strings=['a + b', 'a - b'],
)

would currently accept (a+b)^2 - (a - b)^2 as an answer.

Additionally, let's refactor the validation into a static method.

(I refactored usage of required functions into a static method as part of #32, but am leaving this separate since there would be an actual behavior change.)

Variable Names with Numbers

Some questions very naturally have variable names like phi_n, a_n, etc, where n is an integer. Instead of defining a large number of variables, it would be useful to allow the specification of an indexed variable name.

• Range on the index should be specifiable to some extent. Eg, -infty to infty, 0 to infty, 1 to infty, a to b.
• To make the parsing easier, we should probably require student input in the form a_{n}. This will still allow things like a_{3}^2 to display correctly.
• Sampling will need to be performed after the strings have been parsed to determine which variables are needed.

Relativistic Notation

This is recording some ideas that we discussed regarding relativistic notation.

We want to be able to handle expressions that have implicit sums in Einstein summation notation. Eg: v_mu F^{mu nu}. Because there are dummy indices floating around, these will need to be treated carefully (maybe have a list of dummy indices that are allowed?). Students should enter expressions with all raised or all lowered indices. A metric should be specified so that the grader can compute the various forms of each quantity. Checks on free variables will be necessary.

This will probably need to be a separate plug-in class, RelativisticGrader.

Multivariable functions

We should be able to allow user-defined functions that take in multiple variables. Eg, f(x, y). This will require some extra parsing in calc.py, and will also require some extra checking to ensure that the correct number of arguments are being passed to any given function.

Sibling values and numbered variables

When a FormulaGrader with numbered_vars is used inside a ListGrader, things can break. For example:

from pprint import pprint as pp
from mitxgraders import *
grader = ListGrader(
    answers=[
        ['lambda*hbar*omega', '(ket_{1}-i*ket_{3})/sqrt(2)'],
        ['0', 'ket_{2}'],
        ['-lambda*hbar*omega', '(ket_{1}+i*ket_{3})/sqrt(2)'],
    ],
    subgraders=ListGrader(
        subgraders=FormulaGrader(
            variables=['lambda', 'hbar', 'omega'],
            numbered_vars=['ket']
        ),
        ordered=True
    ),
    grouping=[1, 1, 2, 2, 3, 3],
    debug=True
)

answers = [
    # First pair
    'lambda*hbar*omega',
    '(ket_{1}-i*ket_{3})/sqrt(2)',
    # Second pair
    '0',
    'ket_{2}',
    # Third pair
    '-lambda*hbar*omega',
    '(ket_{1}+i*ket_{3})/sqrt(2)']

pp(grader(None, answers))

When the first input 'lambda*hbar*omega' is evaluated, it also tries to evaluate sibling inputs and pass the values to its comparer function. But when FormulaGrader tries to evaluate the sibling, none of the numbered_vars are defined (since neither the input nor the answer contained any), so an error is thrown.

It's also kind of silly that FormulaGrader is even trying to calculate its sibling values, since none of the comparer_params use the sibling values in this example.

Suggested Fix:

• Only try to evaluate siblings if the comparer_params use sibling values
• Use sibling values to generate numbered variables, too.

Function application post computation

We should add an option to the math grader to apply a function to the result post-testing. By supplying a math function, we could do something like take the sin/cosine of the result, so that the answers are invariant up to phases of 2 pi. By supplying a python function, we could do something like have a python function that tests if the entered number satisfies some condition that can be tested programmatically. This could be the first step in allowing a python function that tests if an entered vector is perpendicular to another vector.

Exponentiation issue in calc.py

There is an issue with calc.py whereby exponentiation doesn't parse in the correct order. I can't remember exactly what the issue is, but I remember that we haven't fixed it.

Debug not available for subgraders

In a listgrader, subgraders cannot set Debug=True. Not sure how we'd even go about trying to make debug output in this situation, but could be helpful.

Generally speaking, debug output for list graders could be improved.

Version 1.2 Checklist

To keep me a bit organized

• Parse and evaluate array entry
• Parse and evaluate multivariable functions
• Custom Comparer Functions for FormulaGrader
• Matrix algebra for FormulaGrader with useful error messages
• #71 Separate class for grading vectors/matrices
  • includes things like adding I, trans, ctrans?dagger, det, trace to defaults
• (Maybe not 1.2.0?): Implement a method to easily extend default constants, a quick opt-in to add Pauli matrices to matrix grader
• #73 Mismatched parentheses detection needs expanding to square brackets
• #76 Refactor abs/norm functions for vectors etc
• Mark off #31 and #44 as appropriate
• #66 Expose multiple comparers (like is_parallel_to, is_eigenvector_of, is_between)
• Review PR #65, which addresses Correlated Grading #46
• Documentation and course examples for MatrixGrader, matrix sampling classes, specify_domain, and usage of sibling variables in FormulaGrader.
• #75 Refactor InvalidInput appropriately
• Update javascript file for any new functions as needed
• Version number, change log, zip file and release!

@jolyonb This stuff seems like a good goal to set before version 1.2. It seems like most of the work is documentation.

Might resolve some of the other issues along the way if they are architecturally convenient.

Matrix samplers

There are a number of special types of matrices that we might want to sample:

Properties:

• (Anti-)Symmetric
• Traceless
• (Anti-)Hermitian
• (Non-)Zero Determinant or Unit Determinant
• Diagonal

Special kinds:

• Orthogonal O(n) or SO(n) (see here and here) (implemented in #186 but blocked due to edX)
• Unitary U(n) or SU(n) (see here) (implemented in #186 but blocked due to edX)
• Upper/Lower Triangular (implemented in #187)

I'm sure there are many, many more possibilities, but the above should keep most people out of trouble! I suspect that most of these are pretty easy to do, too.

Request for variable names with superscripts

See title! Not obvious how you'd implement this...

Mismatched parentheses detection needs expanding to square brackets

What it says on the tin!

Issues with variable name collision when case insensitive

When a user defines a variable E and specifies case insensitive, it's random as to whether e = E or e = 2.78....

Add link to demo course in readme

I think this is it? https://staging.mitx.mit.edu/courses/MITx/jolyon.000/2017_Summer/info

Check for collisions

We should check to make sure there are no collisions between user-defined constants and variables (ie, no constant k and variable k, or worse, defining two constants/variables with the same name). We should also check that user-defined functions don't have duplicated names. We've decided that users should be able to overwrite default constants and functions, however.

Rename mitxgraders/helpers/mitmath to mitxgraders/helpers/calc

• mitxgraders.helpers.mitmath is silly, it has two instances of 'mit'
• and it's long
• Origianlly I wanted it to just be mitxgraders.helpers.math, but that caused naming conflicts in some doctests

Suggestion: use mitxgraders.helpers.calc instead. This would mirror what edX does. Their calc module has structure

calc
|--__init__.py
|--calc.py
|--functions.py
|--preview.py

Refactor abs function to work on vectors

Another thing that came up in comments and I don't want to lose. I saw that norm was added that provides essentially the same functionality, but the abs thinking and notation is definitely standard.

Ideas for the future

(Copied from previous repository)

I was talking with a humanities person about this library. They were very interested in some of the capabilities, particularly for giving feedback, as well as for correlations.

A few ideas came up that we haven't even considered. These are currently just food for thought.

• Same feedback for multiple answers. We presently can describe a response through a dictionary. However, we may want situations like the following:

(
    {'expect':'Obama', 'grade_decimal':1, 'msg':'An interesting choice'},
    {'expect':'Barack Obama', 'grade_decimal':1, 'msg':'An interesting choice'},
    {'expect':'Trump', 'grade_decimal':0.5, 'msg':'Not so interesting'},
    {'expect':'Donald Trump', 'grade_decimal':0.5, 'msg':'Not so interesting'},
)

Here, it would be easier if the expect could itself be a list.

• Correlated answers with all or nothing. Say we have two boxes. We want to give points to a pairing:
  "Mickey" and "Minnie" - scores 1 each
  "Romeo" and "Juliet" - scores 1 each
  "Mickey" and "Juliet" - scores 0 each
  This is presently possible with a singlelistgrader, but not a multilistgrader.

• Correlated answers with combined feedback. Continuing the pairings example above, I want to give feedback if you give me EXACTLY "Mickey" and "Minnie", but not if you only give me one.

handle_node breaks when functions have arguments of different types

A very minimal example:

>>> import numpy as np
>>> np.isinf( np.array([1, 2, 3]), 5 )
Traceback (most recent call last):
TypeError: return arrays must be of ArrayType

I don't completely understand the issue, but apparently np.isinf cannot handle a list where some members are arrays and some members are not. Same issue with np.isnan.

This shows up for us in calc.py, where the self.actions['arguments'] outputs a list of function arguments, some of which might be arrays, some of which might be numbers.

Here's a full example that produces the error:

import numpy as np
from mitxgraders import MatrixGrader, RealVectors

def rot(vec, axis, angle):
    """
    Rotate vec by angle around axis. Implemented by Euler-Rodrigues formula:
    https://en.wikipedia.org/wiki/Euler%E2%80%93Rodrigues_formula
    """
    unit_axis = axis/np.linalg.norm(axis)
    a = np.cos(angle/2)
    omega = unit_axis * np.sin(angle/2)
    crossed = np.cross(omega, vec)

    return vec + 2*a*crossed + 2*np.cross(omega, crossed)


grader = MatrixGrader(
    answers='rot(a, [0, 0, 1], theta)',
    variables=['a', 'theta'],
    sample_from={
        'a': RealVectors(shape=3),
    },
    user_functions={
        'rot': rot
  },
  debug=True
)

(Discovered while writing documentation for specify_domain)

Dependent Sampling

Another idea that may be useful is dependent sampling. Eg, sample x, y, z, then set r equal to sqrt(x^2 + y^2 + z^2) instead of sampling it separately. We'd have to be careful about detecting circular dependencies.

FormulaGrader throws an error when passed an empty formula

This gives an error rather than just grading incorrect.

from mitxgraders import *
grader = FormulaGrader(
    answers="1"
)
print grader(None, "")

MatrixGrader raises error if IdentityMultiple submitted as answer

from mitxgraders import *
grader = MatrixGrader(answers='[1, 2, 3]', debug=True)
grader(None, 'I')
# raises
# AttributeError: 'IdentityMultiple' object has no attribute 'shape'

Raised in MatrixGrader.validate_student_input_shape.

Rename Trace

Instead of tr, require trace. It can render as Tr though.

FormulaGrader should have an `allow_inf` key but it does not

In calc.py, evaluator has an allow_inf key. FormulaGrader should have a configuration option for that.

I think we meant to do this, but it got lost somewhere.

IntegralGrader fails when limits are complex

This should be documented. Perhaps a check in the code should be made as well, and an appropriate error message raised (at least in debug)?

Custom Response Test Script

It would be good to have a way to run through all of the problem files in an edX course, determine which of them contain python scripts, and make sure those scripts instantiate. We could also allow for a special comment to be included in a custom script that our script can then use to test that the grader actually works.

Matrix equivalence grading

We should be able to check how many entries in a tensor expression are correct or not, and assign credit appropriately. At least, it should be an option. Another option that could be enabled is a message that returns which entries are incorrect.

Issue with row*column multiplication

MathArray multiplication behaves ... unexpectedly...for row/column vectors:

a = MathArray([1, 2, 3]) # row: shape(1, 3)
b = MathArray([[1], [2], [3]]) # column: shape (3, 1)

(a*b).size # (1, 1)... a matrix with 1 row and 1 column
(a*b)*b # raises a ShapeError since (1,1) * (3, 1) does not work.

Relevant: MathArray mostly uses np.dot for multiplication, the documentation for which is very good: https://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html (Note that the documentation is for v1.13, we use 1.6. But I don't think dot has changed. It references matmul as preferred over dot, which does not exist in 1.6)

Suggested fix: MathArray's __mul__ method has some checks to see if self or other has ndim=0. These checks should actually be self or size has size=0.

Documentation for numpy.ndarray.size:

https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.size.html

Division by zero

We should probably have some special error catching for division by zero in calc.py. At the moment, students get a "Could not parse your formula" error, which is not very helpful.

is_callable_with_args can be inaccurate with kwargs

For normal functions, is_callable_with_args calls get_number_of_arguments, which includes positional arguments AND keyword arguments. For numpy ufuncs, is_callable_with_args is probably accurate. (It relies on ufunc.nin, which seems to be the numpy of positional inputs.)

We are primarily using is_callable_with_args for validating the number of positional arguments in student input, so we could change it to be positional-only.

Relevant: https://stackoverflow.com/a/197053/2747370

(In particular, this came up with np.transpose, which is a normal function with signature transpose(array, axes=None), so using np.transpose as a user-function does not work. Of course, a quick fix is to just use lambda x: np.transpose(x) instead.)

Separate class for grading vectors/matrices

Related to #70

I suggest creating a subclass of FormulaGrader for grading vectors/matrices:

class ArrayFormulaGrader(FormulaGrader):
  # custom configuration schema
  # custom default functions and variables
  # shadows check_answer

Reasons to create a new ArrayFormulaGrader class for grading vectors/matrices:

• Have separate default functions and variables.

  • I for Identity: ArrayFormulaGrader could have a default variable named I for the identity operator.

    If FormulaGrader has a default variable named I, it is going to be useless most of the time, AND if authors try to overwrite it, they'll get warning errors (which can be suppressed, but still annoying).

  • Conceptually, it also makes sense to only provide transpose and adjoint functions when dealing with matrices. However, naming conflict is very unlikely for these.

• Separate Default Configuration: Currently, vectors and arrays are currently enabled through a max_array_dim configuration key. This can be hard-coded to 0 (scalars only) for FormulaGrader, and configurable for ArrayFormulaGrader (with a default of 2, for matrices). Not a big deal, but I find it conceptually nice.

• Customized Error Handling: Currently, whenever something goes predictably wrong with matrices, students get an error NOT marked incorrect. Examples:

  • multiplying / adding matrices with a shape mismatch
  • raising non-square matrix to powers

  I think that raising errors in this situation makes sense as a default. However, I can easily imagine a linear algebra course where the concept being assessed is shape mismatch. In that case, it might make sense to mark student incorrect for shape mismatch rather than raise an error.

  ArrayFormulaGrader could have configuration keys for this, easily implemented by shadowing FormulaGrader.check_answer and catching the appropriate exceptions.

  Of course, we could do this directly in FormulaGrader, but that class is, in my opinion, a bit bloated already.

Things to implement in the example course

• Factorial function working on a random integer, for example, in the series expansion of exp
• Error messages with mismatched parentheses, unknown variables/functions (including wrong case!)
• Metric suffixes being removed
• A bunch of examples from the documentation
• Sampling when sqrts go negative
• LaTeX2edX code samples

Example Course Update

We should add in to the formula grader examples that division by zero is flagged, as are out-of-domain functions and overflows.

Keeping python_lib.zip up-to-date

After merging a few recent PRs, I don't think python_lib.zip is up-to-date anymore. (E.g., #15).

What's our general philosophy on this? Should the primary author of PR not worry about it, and it's the merger's responsibility to update python_lib.zip just before merge? Could we possibly add a test that checks if python_lib.zip is up-to-date, so that travis warns us if we forget?

EdX Errors with Python Newlines are Problematic

Of course Python newline (\n) doesn't display correctly if used in edx XML. But if you use one in an error message, you get a traceback, too!

def cfn(e, a):
    raise Exception("A \n B")
]]>
</script>

whereas with HTML newlines only:

def cfn(e, a):
    raise Exception("A <br/> B")
]]>
</script>

So ... that's weird.

Not a big problem for us, easily resolved by editing AbstractGrader's __call__ method.

Refactor InvalidInput

Incorporate it into StudentFacingError appropriately. This came up in comments, but I figured I'd put it into an issue to keep track of it.

Parser doesn't work properly

The tensor parser broke the old parsing. The following no longer works:

import mitxgraders
evalf = mitxgraders.helpers.calc.evaluator
print(evalf("epsilon_max", {"epsilon_max":2}, {}, {}))

The error is raised by pyparsing; the old variable names with underscores aren't being accepted.