stevenclontz / checkit

Platform for authoring free and open randomized exercises for practice and assessment.

CheckIt Platform

A platform for authoring free and open randomized exercises for practice and assessment. Includes the Python/Sagemath CheckIt Dashboard for authoring and generating random exercises, and the TypeScript CheckIt Viewer for publishing exercise banks online.

python -m pip install --upgrade pip
python -m pip install --upgrade checkit-dashboard

Homepage at checkit.clontz.org.

Documentation for authors and developers is available in the wiki on GitHub.

Package Development

Python

Development uses pyenv:

https://github.com/pyenv/pyenv

with pyenv-virtualenv:

https://github.com/pyenv/pyenv-virtualenv.

Run the following, replacing PYTHON_VERSION with the version defined in the platform/src/checkit/static/PYTHON_VERSION file.

pyenv install PYTHON_VERSION
pyenv virtualenv PYTHON_VERSION checkit

The virtual environment for this project can be activated automatically in directories with a .python-version file with the contents checkit by following instructions at https://github.com/pyenv/pyenv-virtualenv#activate-virtualenv. Or use pyenv activate checkit and pyenv deactivate to do this manually.

Now to install the in-development package into the virtual environment.

pyenv virtualenvs # should show `* checkit` (note the `*`)
python -V # should show value of PYTHON_VERSION
python -m pip install --upgrade pip
python -m pip install -e dashboard[dev]

To enable this virtual environment for use as a Jupyter kernel for the dashboard:

python -m ipykernel install --user --name=checkit --display-name "CheckIt Platform"

Build & deploy package

Make sure versions are set as intended! Then...

cd dashboard
python update_viewer.py
rm -rf dist/*
python -m build
python -m twine upload dist/*

Updating docs

Just python build_docs.py.

checkit's Issues

handle caching of generated exercises more carefully

Support generating full solutions in addition to brief answers.

be consistent with using Elements vs ElementTrees

allow generation of "essay" vs "file upload" Canvas question bank

create proper package

add instructions for broken cocalc links

Default behavior should be random

By default, downloading and running this repository should produce random exercises, not exercises relying on a fixed seed. Not only is this what most people will expect, but it is a "fail safe" approach -- users should not accidentally produce problems identical to some other possible public set.

configurable presentation at build time

E.g. let a build specify if matrix delimiters are parentheses or square. Basically, providing access to https://doc.sagemath.org/html/en/reference/misc/sage/misc/latex.html at build.

bank-pretext.ptx export along with build

Build pretext file with first 5 seeds of bank that can be directly <xi:include/>d into a PreTeXt document.

Support arbitrary languages for generating exercises

While I want to support Sage as the default language for authoring exercises, the logic of the CheckIt platform is just Python. So if I factor that out, then I could support arbitrary languages for authoring exercises (Ruby, JS, Perl, etc etc).

Format helpers for certain expressions

e.g. show_difference(-2,-3) == "-2+3"

distribute JSON of randomized data with bank JSON blob

merge viewer project into here

Print roots as surd

When you type sqrt(x) in Sage, it will print with the $\sqrt$ symbol, but if you want $\sqrt[3]{x}$, sage will convert it to $x^{1/3}$.

I hacked a little function to print surds. It needs error validation and test, but we could implement it in a common.sage type file like you have in tbil. The function seems to work with the dashboad.

def surd(var='x',a=2,b=1,):
    if len(var)>1:  #We put parentheses if var contains more than one term
        var="("+var+")"
    return r"\sqrt[%d]{%s^{%d}}" % (a,var,b)

Include outcome description in canvas outcome export

Whitespace

Right now whitespace between

tags is being preserved in LaTeX output. This shouldn't happen as whitespace is used for paragraphs in LaTeX.

D2L/brightspace rendering issue

I've imported the CheckIt problem database into Brightspace. The V1 standard on verifying vector space axioms tends to display the extra space "," characters, which confused some of my students as they thought they had to divide stuff by other stuff. (Oops! I hadn't realized in enough time to send them a warning.)

Not sure why this happens but Drew sent me here to log my report.

Generate Unique IDs in Canvas outcomes csv

Ran into an interesting error with the Outcomes CSV. For some reason, I could not get some of my outcomes to upload -- it kept throwing an error. Turns out the error message gets emailed to you:

`Hello Lewis, Dr. Drew M.,

Your outcomes were successfully imported, but with the following issues (up to the first 100 warnings):

Row 2: Outcome with ID tbil-la_00_E1 already exists in another unrelated course or account (MA-238-101: Differential Equations I)
Row 3: Outcome with ID tbil-la_01_E2 already exists in another unrelated course or account (MA-238-101: Differential Equations I)
Row 4: Outcome with ID tbil-la_02_E3 already exists in another unrelated course or account (MA-238-101: Differential Equations I)
Row 5: Outcome with ID tbil-la_03_V1 already exists in another unrelated course or account (MA-238-101: Differential Equations I)

You can now manage them at https://usaonline.southalabama.edu/courses/7134/outcomes

Thank you,
Instructure`

In particular, MA-238-101 is @StevenClontz 's course, and he presumably had these outcomes as testing the implementation of the Canvas Outcomes CSV. It seems that we need to generate unique IDs, or two instructors at the same institution may run into trouble.

Workaround: I created different IDs by hand for my course.

Matrix generator improvements

By default (but perhaps this is configurable), generated matrices should not include zero-rows, 0=1 rows (if augmented), or repeated rows.

Remove blank lines in LaTeX displaymath

LaTeX does not allow blank lines in displaymath. However, if one is not careful, the template can generate this. Example: https://github.com/StevenClontz/checkit-tbil-la-2021-dev/blob/65dc25b4f6c45a4d0992aa8701b984a19665e614/outcomes/LE4.xml#L12

Code:

Consider the following
{{#system}}
system of linear equations.
{{/system}}
{{#vec_eq}}
vector equation.
{{/vec_eq}}
      <me>
          {{system}}
          {{vec_eq}}
      </me>

If system or vec_eq is not defined (which should always happen here), this inserts a blank line before the vector equation, which causes an error in compiling LaTeX. This can probably be fixed by the user here by

<me>
       {{system}} {{vec_eq}}
</me>

but it would be ideal for Checkit to, whenever it encounters <me>, gobble up any blank lines.

Calculator / Sage Cell Modal

Add a handy button when viewing exercises that reveals a convenient calculator or Sage cell on the screen for calculations.

Include metadata in all output formats

Currently we don't build things like Outcome slug/title or Seed number into all output formats. We should do this, keeping in mind that instructors may not wish to display this information (e.g. make sure it's wrapped in a LaTeX environment that can be shown/hidden).

Jupyter GUI for Dashboard

Replace code cells with a GUI interface

Handle False data JSON on Python side

If a random exercise's JSON has a false value, then Python should eliminate that element from the data XML.

`ol` shouldn't be hardcoded for letters

external tools

Allow authors to specify external web tools (presented as iframes for HTML, links for LaTeX) for use in exercises.

LMS export

• canvas
• d2l
• moodle
• WebWork
• blackboard
• others?

set "context" in dashboard

by setting a "context" on the dashboard, the build changes.

Inject build information into LMS exercises (all private exercises?)

List numbering supressed in PreTeXt (sometimes)

Apparently if you give PreTeXt an <ol> but the contents of the <li> are not enclosed in <p></p> (e.g. if you just give it a <me>), it will not display the numbering. The solution is to enclose the <me> in <p>, which seems to make PreTeXt happy.

Currently, CheckIt appears to supply the needed <p> in the HTML output, but not the PreTeXt output. A workaround is for the user to ensure that a <p> is there in the template (though note that a user using the HTML output to debug will not notice an error!).

Example outcome that will show this bug: https://github.com/StevenClontz/checkit-tbil-la-2021-dev/blob/main/outcomes/LE1.xml

add "restart kernel" to dashboard instructions

Fix leading negatives in linear equation display

See TeamBasedInquiryLearning/checkit-tbil-la#12

Add <table/> support

Canvas prompts do not display standard

Many instructors (i.e. me) like to display the standard number on assessments so that students know which problems they can skip. However, when a Canvas quiz is created and pulls a question from the question bank, the displayed title is simply "Question 1" or similar. It's not clear if this is a Canvas "feature" or a feature of the QTI standard.

CheckIt VS9 weird variable issue

Checkit generated this system of homogeneous equations for VS9. Notice the variables are x_1, x_2, x_3, but then x_1 re-appears. The Answer key for checkit assumes the second copy of x_1 is x_4 and generates a matrix with five columns:

Canvas equation server issues

In 54de511 I fixed how equation URLs for canvas are generated. However, we should use the https://canvas.instructure.com/equation_images/ endpoint instead in case its implementation (e.g. that AWS server) is changed. I think to do this, I need to double-encode the URL.

First-class support for systems of equations

Support print statements in generators

See StevenClontz/checkit-platform#1

port checkit-tbil-la helpers

Matrix generator

See TeamBasedInquiryLearning/checkit-tbil-la#7

Given dimensions and number of pivots, generate random matrix where:

• uses integers between -9 and 9 for itself and its RREF
• doesn't have repeated columns
• doesn't have a column of zeros

Add build info to LMS export

Add a timestamp to LMS question bank titles so instructors know which builds they have imported (public/2020-01-15/etc.)

Native Canvas MathJax

@StevenClontz noticed that you can get to Canvas's MathJax via for example

<img class="equation_image" src="/equation_images/codeGoesHere" />
or
<img class="equation_image" src="/equation_images/z'%253DR-%255Cfrac%257BRz%257D%257Bv%257Du%255Cleft(t-a%255Cright)%255Chspace%257B1em%257Dz(0)%253D0" />

So it looks like for example, %253D is the ecnoding of %3D which is =. I believe the python library urllib can be used to automate this https://docs.python.org/3/library/urllib.html . This would enable the generated QTI to use the native Canvas MathJax, instead of the external website CodeCogs as it does currently. Perhaps we could leave a flag for the user to select CodeCogs if they wanted, but relying on an external website adds an additional point of failure.

Un-support invalid PreTeXt

The source at https://github.com/TeamBasedInquiryLearning/checkit-tbil-la/blob/main/V3.ptx#L4 is invalid PreTeXt:

I'm guessing a <claim/> inside a <p/> is bad, and text nodes in a <claim/> (not organized with a <p/>) are also bad. So we should break our support for this in checkit, and then fix the tbil-la bank to use valid PreTeXt instead.

Refactor how seeds are chosen.

Moving forward, seeds 000-999 will be considered "public", to be used in contexts such as https://checkit.clontz.org. The seed value should be distributed alongside the built exercises.

Seeds 1000+ are reserved for instructor use, so students cannot simply look up the solution from the public library.

Mathml font size

https://twitter.com/TChihMath/status/1399806815064494094?s=20

Image support

• support passing SageMath Graphics objects which are converted to ???
• support user-created banks of images that can be inserted via
  • URL [BASE_URL]/images/[outcome_slug]/[filename]
  • local folder for TeX images/[outcome_slug]/[filename]

correctly change working directory

For reasons we have to change the working directory when importing certain Sage files. However, if there's an error while the directory is changed, the change is never reverted, causing things to break in notebooks (until restarting the kernel).

While this shouldn't happen in production, we should use a try/finally workflow (or context manager e.g. https://gist.github.com/nottrobin/3d675653244f8814838a ) to fail more gracefully.

V4 issue

Here's another D2L/brightspace display issue. It might have been patched in the previous patch, but for completeness here it is.

don't overwrite docs/index.html if it exists

Useful for customizing bootstrap theme

build student-only viewer as an option