guesswho1234 / rex

Shiny dashboard for creating and evaluating written single and multiple-choice exams.

Rex: Shiny R/exams Dashboard

This repository provides an interactive shiny dashboard for creating and evaluating written single and multiple-choice exams. The underlying workhorse is R/exams, in particular the so-called NOPS exams functions which support both single-choice and multiple-choice exercises along with automatic evaluation.

Rex is developed by Sebastian Bachler and is still work-in-progress.

Run

Localhost

Install all the necessary R packages (see init.R) including their dependencies.

Start up via shiny::runApp(appDir = 'base directory containing app.R', host = '127.0.0.1', port = 8180); or, alternatively, via RStudio by opening and running app.R.

Heroku

Install the following buildpacks in the following order on your heroku app:

• https://github.com/rricard/heroku-buildpack-dpkg.git
• https://github.com/virtualstaticvoid/heroku-buildpack-r

Set the following environment variables (called "Config Vars" on Heroku):

• INCLUDE_DIR: /app/.dpkg/usr/include/poppler/cpp/
• LD_LIBRARY_PATH: /app/.dpkg/usr/lib/x86_64-linux-gnu/:/app/R/lib/R/lib:/app/tcltk/lib
• LIB_DIR: /app/.dpkg/usr/lib/x86_64-linux-gnu/

Deploy this repository to your Heroku app and make sure the web dyno is turned on.

Note

This setup was tested on the "heroku-22" stack.

Note

Heroku shuts down your application if you exceed the memory allocated to your dyno type. If you are using the "Eco" dyno type, the application also shuts down when there is no traffic for 30 minutes.

Warning

While most features of Rex can be used with the low tier dyno types Heroku offers, evaluating exams cannot. Therefore, make sure to switch to high performance dyno types if you wish to evaluate exams with Rex hosted on Heroku.

Docker

Build your image with the supplied docker file: docker build --platform linux/x86_64 -t shiny-docker-rex ..

Run your image: docker run -p 8180:8180 shiny-docker-rex.

Use

Once you got the application running, authenticate using "rex" as both login and password. This then brings you to the main application. At the very top, you will find a navigation bar with the following functionalities:

• Rex logo (click to get to the top of the page)
• "EXERCISES" tab where exercises are managed
• "EXAM" tab where exams are created and evaluated
• "TOOLS" tab where any additional addons are located
• "TeX mode" button to enable or disable LaTeX input mode (when turned on, fields where LaTeX is accepted will appear with a pink border)
• "Hotkeys" button to enable or disable all hotkeys for the application (when hovered, overlays buttons with the corresponding hotkey mapping)
• Switch for displaying buttons with icons or text
• Switch to change the application language
• Heart icon to indicate the application connection status
• "Log out" button to log out

At the very bottom you find all the information regarding source code and licensing.

Workflow

Manage exercises

The entire management of exercises takes place in the "EXERCISES" tab of the application. For all exercises the app distinguishes between simple editable exercises (highlighted with purple accents) and complex not editable exercises. Based on that, various functionalities are enabled or disabled.

Exercise functionalities:

• Change seed value used for random processes in complex exercises
• Create new simple exercise from scratch.
• Import one or multiple existing simple exercises and complex exercises either as RNW or RMD files
• Save exercises
• Convert complex to simple exercises
• Parse exercises (i.e., check if everything is still fine)
• Mark exercises as examinable / add them to be considered for the exam
• Remove exercises
• Filter exercises
• Reorder exercises
• Group exercises into blocks to equally draw from when creating exams

Beyond that, as mentioned above, simple exercise can be edited directly within the application. When editing an exercise, the following fields can be edited:

• Name
• Author
• Type
• Points
• Section
• Question (accepts LaTeX code when "TeX mode" is turned on)
• Figure
• Answers (consisting of the solution, text, and a solution note)

Data of fields with a yellow background shade will generally be observable by exam participants. Data of fields without this yellow background shade will never be observable by exam participants. Once a field is edited, the exercise is automatically market with a red warning and, therefore, requires to be parsed again before it can be saved or marked as examinable.

Additionally, on the top right of the exercises tab, summary information on all exercises that are marked as examinable is shown.

Create exams

To create exams, navigate to the "EXAM" tab and further to "Create exam". There you will find one large form with various options to set. The following fields are available in this form:

• Exam date
• Seed (used for randomization of exercises, answers s well as random processes within exercises)
• Number of scramblings
• number of exercises
• Fix exercise order
• Fixed points per exercise
• Registration number digits
• Show points
• Duplex printing
• Replacement evaluation sheet
• Avoid wrap in choice list
• New page per exercise
• Language
• Institution
• Title
• Course
• Intro (accepts LaTeX code when "TeX mode" is turned on)
• Number of blank pages
• Additional Pdf pages

Input fields that are empty do not require values and are optional. All other fields do require explicit input values and are not optional. While initialized with the value 0, obviously, the "number of exercises" must be greater than 0 in order to be able to create an exam. To set a different number than 0, however, requires exercises to be marked as examinable. In other words, the maximum number of exercises in an exam is the number of exercises marked as examinable.

When pressing the "Create exam" button the exam creation process starts. When this process ends, a popup will show and, given that no errors where encountered, offer to save the exam. When saving the exam, a ZIP archive will be downloaded. In this archive you will find the following files:

• A PDF file for each scrambling including the evaluation sheet with the institution title, and course text, the intro text, set number of blank pages, and any additionally added PDF pages
• A HTML file for each scrambling including the solutions and solution notes
• A RDS file which is needed to evaluate the exam later
• All of the exercises used in the exam
• A TXT file "input.txt" containing all the input values used for creating the exam

Evaluate exams

To evaluate exams, navigate to the "EXAM" tab and further to "Evaluate exam". There you will find another large form with various options to set. The following fields are available in this form:

• Fixed points per exercise
• Registration number digits
• Partial points
• Rule for point deductions
• Negative points
• Grade exams
• Grading key
• Language
• Rotate scans
• Solutions
• Registered participants
• Evaluation scans

Input fields that are empty as well as the "Grading key" fields when "Grade exams" is not checked do not require values and are optional. All other fields plus the fields "Solutions", "Registered participants", and "Evaluation scans" do require explicit input values. Specifically:

• "Solutions": the supplied RDS file in the ZIP archive downloaded when the exam was created
• "Registered participants": a CSV file containing all the information of participants
• "Evaluation scans": all the evaluation sheet scans as either PDF or PNG files

When pressing the "Evaluate exam" button the exam evaluation process starts. In a first stage, all the supplied scans are processed. Once this first stage is finished, a popup will show and, given that no errors where encountered, allow to inspect and edit all the scans together with the information extracted from them. When proceeding, the second and last stage starts. Once also this stage is finished, again a popup will show and, given that no errors where encountered, offer to save the exam evaluation. Also, it is possible to revert back to the first stage again. When saving the exam evaluation, a ZIP archive will be downloaded. In this archive you will find the following files:

• Two ZIP archives, one containing the scans as PNG files matching what was supplied for the "Evaluation scans" field (ending with "_nops_scan.zip") and another containing the evaluation documents of the participants (ending with "_nops_eval.zip"). For each participant the evaluation document is further placed in a separate folder named after what was supplied in the "id" column of the file supplied for the "Registered participants" field
• Two CSV files, one machting what was supplied for the "Registered participants" field and another containing all the evaluation data (ending with "_nops_eval.csv")
• One RDS file matching what was supplied for the "Solutions" field
• A TXT file "statistics.txt" containing some basic evaluation statistics of the exam
• A TXT file "input.txt" containing all the input values used for evaluating the exam

Addons

Addons add custom functionalities to the application, which do not necessarily tie into the standard workflow described above.

Contribute

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

rex's Issues

Allow only mchoice questions for nops exam

install newest version of rexams when building

Improve key input functionality when editing tasks.

In case of websocket disconnect change appearance

Restructure exam evaluation

• Display daten.txt and highlight scans with problems or where no matching id was found in the uploaded file with registered participants
• Clicking on lines (from daten.txt) allows editing of individual evaluations (side by side view of scanned image and a html representation of the evaluation sheet)
• Finalizing passes / creates / updates all files for nops_eval (daten.txt, png scans, registered participants, solutions)

Fix file write in javascript

Writing*.rnw and*.csv files adds empty row after each text row (see f.e. when writing task files).

highlight scan evaluation as red if parameters not valid

• registration is set
• scrambling is not NA
• type is not NA
• sheet is set

create secure login bevore having access to tool

Include PNG file input field for editable task

Allow to include one PNG file per editable task.

add hotkeys for inspecting scans

add zoom for scan inspect

add stats to scan evaluation

• number of valid
• number of invalid
• number of not assigned

add fontawesome library as file and get rid of cdn load

Handle duplicated scans

Export task

exporting task sometimes gives htm file instead of rnw

style trycatch responses in modal

Fix languages on heroku

On heroku latex throws an error if something other than "en", "ja", "ko", "tr" is selected for the language when creating an exam

put prepare functions for exam creation and evaluation into background

when change is made to editable task blocks are not deactivated properly

exam state etc. is deactivated
forgot to deactivate blocks also

Fixed points for questions in evaluation

by default hide yellow scan rows in scan evaluation (option to show)

add accept & next error button (space bar) to go to next item with errors

Export all tasks

Download fires infinitely when a click on the download button is simulated.

put scan inspect below item

add tooltip for rules

Find better names for rules (maybe add explanation tooltip).

only process scans without errors

pad zeros to reg ids

sort scan evaluation by statusClass first and registration second

Prevent duplicated names when evaluating exams

Evaluating the same exam (same file name for solutions file - *.rds) causes problems due to duplicated names.
As a result incorrect files could be used when evaluating the exam.

fix objectmodel['e'] for tasks that had no issues

see todo comment in script.js

disable logout during wait

loading indication for scan compare table

multiuser compatibility

• avoid sessions interfering with each other
• sometimes directories are not unlinked properly

Some hotkeys are not properly deactivated when editing tasks

Shift+F moves to search field even when editing fields of editable tasks.

make repo public

Add field validation and helpers to fill in forms

Examples:

• All exam tasks require to have the same amount of choices
• All exam tasks need at least 2 choices
• All choices need to have unique values
• At least one exam task needs to be selected

allow in eval to set mark to false

for red highlighting of scan evaluation check for valid numbers instead for NA

if eval finalize fails, circle back to scan eval

add additional exam input fields

• - reglength
• - replacement
• - samepage
• - newpage

Sanitize editable field text inputs

Remove for example all html tags from content put into editable text field

Validate editable task inputs

for example when editing points, check if its an integer number. if not, set points to 1.

check pad zero after scan inspect edits - sometimes "0" adn "00" for scrambling field in output file

Improve IuF tasks

Add:

• library(exams)
• library(iuftools)

Replace if(MAKEBSP) set.seed(1) to if(exists(MAKEBSP) && MAKEBSP) set.seed(1)

error when scan fields are corssed out (type, scramblin exam id)

Remove question type field from task view

remove field from gui, but keep it in object model. also only allow tasks with "mchoice" to be added to the list in the first place. if not "mchoice" display error.

Properly close file connections

• Unlink all files when not needed anymore
• Maybe even on.exit(unlink(...)) right after tempdir() or tmpfile()?
• Or unlink all at the end?

add transparency to rex favicon

Install tex live on heroku during build

Installing tex live during runtime takes very long. Therefore, it would be better to install tex live already when deploying / building.

check if exam id, scrambling and type is valid

read solutions file and populate fields in the same manner as the registration field (drop down with autocomplete)

if the scanned value does not match any entry -> mark row as red