Coder Social home page Coder Social logo

learnrhash's Introduction

learnrhash

R build status Lifecycle: experimental

Installation

This package is still in the early stages of development and currently is only available from GitHub. To install the development version run the following:

# install.packages("devtools")
devtools::install_github("rundel/learnrhash")

Usage

This package is meant to provide addition tools for collection student answers to learnr tutorials. The package does not transmit the answers in any way, but instead provides a convenient method to generate a compressed text based representation that can be easily copied and pasted. The students can then submit these solutions by pasting this “hash” into an online web form (e.g. Google Forms, Microsoft Forms, etc.) or a learning management system quiz or assignment tool.

To enable this functionality, all you need to do is include the following in a learnr Rmd document:

## Submit

```{r context="server"}
learnrhash::encoder_logic()
```

```{r encode, echo=FALSE}
learnrhash::encoder_ui()
```

which results in the Submit topic appearing in the tutorial with all of the necessary shiny logic and ui inserted, as shown below.

Encode solutions

In the example above a url for http://localhost given, this value can be replaced with whatever link you would like to use for submission. All the students will need to do is to paste the generated hash into a text response field on whatever web form you choose to use.

Working with Hashes

The expectation is that after students submit their solutions you will be able to obtain some tabular representation of these results that can be read into R as a data frame. The package includes a simple example of this type of data which is loaded as follows

example = readRDS(system.file("example.rds", package="learnrhash"))
example
## # A tibble: 2 × 3
##   student student_id hash                                                       
##   <chr>        <dbl> <chr>                                                      
## 1 Colin        20000 QlpoOTFBWSZTWeVuJ2oAA0d/gP/7aAhoC7BViyIOyr/v/+BAAcACsAS7C1…
## 2 Mine         10000 QlpoOTFBWSZTWYeyPVYAA0x/gP/7aAhoC7BVgyIOyr/v/+BAAcACsAdqC1…

Currently the package provides two functions for extracting question solutions and exercise solutions from these hashed data, for both functions the only required argument is the name of the column containing the hashed solutions

To extract the all submissions use,

learnrhash::extract_hash(example)
## # A tibble: 10 × 5
##    student student_id id            type                data            
##    <chr>        <dbl> <chr>         <chr>               <list>          
##  1 Colin        20000 code          exercise_submission <named list [4]>
##  2 Colin        20000 code2         exercise_submission <named list [4]>
##  3 Colin        20000 details       question_submission <named list [4]>
##  4 Colin        20000 not_a_planets question_submission <named list [4]>
##  5 Colin        20000 planets       question_submission <named list [4]>
##  6 Mine         10000 code          exercise_submission <named list [4]>
##  7 Mine         10000 code2         exercise_submission <named list [4]>
##  8 Mine         10000 details       question_submission <named list [4]>
##  9 Mine         10000 not_a_planets question_submission <named list [4]>
## 10 Mine         10000 planets       question_submission <named list [4]>

and to get just the exercises,

learnrhash::extract_exercises(example)
## # A tibble: 4 × 5
##   student student_id id    type                data            
##   <chr>        <dbl> <chr> <chr>               <list>          
## 1 Colin        20000 code  exercise_submission <named list [4]>
## 2 Colin        20000 code2 exercise_submission <named list [4]>
## 3 Mine         10000 code  exercise_submission <named list [4]>
## 4 Mine         10000 code2 exercise_submission <named list [4]>

or questions,

learnrhash::extract_questions(example)
## # A tibble: 6 × 5
##   student student_id id            type                data            
##   <chr>        <dbl> <chr>         <chr>               <list>          
## 1 Colin        20000 details       question_submission <named list [4]>
## 2 Colin        20000 not_a_planets question_submission <named list [4]>
## 3 Colin        20000 planets       question_submission <named list [4]>
## 4 Mine         10000 details       question_submission <named list [4]>
## 5 Mine         10000 not_a_planets question_submission <named list [4]>
## 6 Mine         10000 planets       question_submission <named list [4]>

If you would like to experiment with this decoding and extraction while writing your tutorial you can also include decoding logic and ui elements into the tutorial in a similar way that the encoder was included. Simply add the following lines into your Rmd,

## Decode

```{r context="server"}
learnrhash::decoder_logic()
```

```{r encode, echo=FALSE}
learnrhash::decoder_ui()
```

Decode solutions

learnrhash's People

Contributors

gadenbuie avatar matackett avatar rundel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

learnrhash's Issues

Downloading hash

Is there an easier way for students to download the submission hash?

Right now, I have two text boxes underneath the "Generate Submission" box that asks for students' names and hashes. The idea is that they enter their name into the first box and copy-paste their hash into the second box. Next, they will press "Submit", which creates a dataframe of 2 variables: their name and their hash. Lastly, they will press "Download," which downloads an .rds file that is ready to be uploaded to Canvas.

Is there any way to simplify this? For example, for there to be a button in which students can generate and directly download the submission hash in one step? I am concerned that students could copy and paste each others' hashes.

base64 strings problem

I'm trying to implement learnrhash in a learnr tutorial with gradethis. I added the chunks to the generate the hash and got the following error when trying to extract from it:

learnrhash::extract_questions(foo, hash = hash)
Error in base64enc::base64decode(txt) : I can only decode base64 strings

I'm certain I'm on a 64 system. I also tried to extract using the gui code, but got a set of different errors. Has this error come up for anyone else?

Long hash

A student tester completed one of the tutorials I wrote with learnrhash. It's a long tutorial, as we are developing them to correspond with chapters of a course textbook. However, the hash at the end was quite long, to the point where the student's RStudio froze and crash because they tried to copy such a long string.

Is there any way to shorten the hashes generated at the end?

And again, really appreciate your work! We're testing with students to iron out any issues now before rolling out to a class of 100-200 students at Harvard in the coming weeks.

Issue simply replicating your example and run successfully the two functions to extract results

Hi,
I simply tried to replicate your example and see how it works. However, after installing the package and create the objective example:

example

A tibble: 2 × 3

student student_id hash

1 Colin 20000 QlpoOTFBWSZTWeVuJ2oAA0d/gP/7aAh…
2 Mine 10000 QlpoOTFBWSZTWYeyPVYAA0x/gP/7aAh…

I run into this error:

learnrhash::extract_questions(example, hash)
Error: Problem with filter() input ..1.
ℹ Input ..1 is type == "question".
x object 'type' not found
Run rlang::last_error() to see where the error occurred.

I am a bit confused of what the error on my end is. It seems I am applying the function correctly. Any advice is appreciated.

Hash extraction fails

The commit aca8e78 that was meant to fix parts of #17 seems to revert parts of the previous fix for extraction, causing it to fail. With the latest learnr devel, learnrhash::extract_hash fails due to a missing "id" column.
In addition, from what I can tell, the "exercise_submission" type used for extract_exercises was the old name (and similar for questions); the previous commit for fixing extractions changed it to "exercise" and "question", which was fine up until this latest "fix".
So it looks to me that the latest fix regressed to some old learnr format, rather than updating it to the latest.

I'm now trying to force my quizzes to load the previous learnrhash version, and that still works with the latest learnr devel version.

Copy-pasting hash with clipr

I was wondering why clipr doesn't seem to work with the current encoding UI, and if you know of any copy-pasting alternatives.
Also, thank you again for your work on this package! We will most certainly be using it in the data science course I'm helping my professor to develop:)

Can’t generate hash

My tutorials are all generating errors when I go to generate a code. I’ve been playing around with it all day. I even downloaded a minimal example (demo_min.rmd) and I get the same error. It says that my content=server needs to be in the same chunk, but it is.

The name of the authors is not filled in the DESCRIPTION file

Hi Colin!

I used the citation() function to get information to cite learnrhash but seems like you need to add the name of the authors in the DESCRIPTION file in order to work correctly. I'll correct the citation manually in my .bib file, but I think is a good idea to fill the description so is easier for other people to cite the package :) Thanks

> citation("learnrhash")

To cite package ‘learnrhash’ in publications use:

  Who wrote it (2021). learnrhash: What the Package Does (Title Case). R package
  version 0.1.0.9000.

A BibTeX entry for LaTeX users is

  @Manual{,
    title = {learnrhash: What the Package Does (Title Case)},
    author = {Who wrote it},
    year = {2021},
    note = {R package version 0.1.0.9000},
  }

ATTENTION: This citation information has been auto-generated from the package
DESCRIPTION file and may need manual editing, see ‘help("citation")’.

image

Problems with encoder

There seems to be a discrepancy between the help for encoder_ui() and what is stated on the github site (see below) about how to use it. In particular, the help in R for this package says we need to specify ui_before() and ui_after() etc. When I run the code below, I get a page long hash and no "COPY" button.

Submit

learnrhash::encoder_logic()
learnrhash::encoder_ui()

Failure mode to be avoided

Recall that I implemented a variation on learnrhash and have been using it in a large course at the US Air Force Academy. The main difference is that my hash code gets constructed by monitoring events in the tutorial session.

After having nine tutorial assignments for each of 160 students, it's become apparent that there is a user behavior that triggers unanticipated and unwanted behavior by the overall system. The problem occurs when a student does not complete the entire tutorial in one session, perhaps because he or she went off to do something else or because the server failed, etc. Often, the student has not yet submitted the hash code to the course support software.

When the student returns to the tutorial in a new session, the previous answers have been cached in the browser and re-appear to the student. However, these previous answers have not triggered a learnr event in the current session and so aren't encompassed by the hash code. (I think, but am not certain, that this will also happen when the hash is being constructed by polling the tutorial state on the server.) The student submits the current session's hash code, understandably believing that this reflects that state appearing in the student's document, which includes events generated in a previous session. But the hash only reflects events in the current session. Result: the student claims that they have submitted all their answers but in reality only a few were represented in the hash code sent to the course support system.

For approximately 1/3 of my students, this has resulted in a serious undercount of their answers in each tutorial document. So it's a mainstream behavior for my students.

We tried to train the students to submit the hash code more often, but this did not alleviate the problem. Lacking a proper solution, I've tweaked the learnr code to change the behavior when a correct answer is submitted. The original behavior is to show the feedback message and no other control. In my tweaked version, a "Refresh submission" button appears under the feedback message. That way, a student can go through his or her completed tutorial in a new session and quickly transfer the browser-cached correct answers into the hash code for that session. See this tutorial for an example.

I'm posting this issue so that you can avoid this situation and, if you find a better solution, can tell me about it. Alternatively, I might make a pull request to {learnr} to implement the "Refresh submission" behavior as an option.

Do hashes actually need to include outputs?

Hashes are quite large. Manually decoding the generated hash shows that it's storing a lot of data, including the outputs for exercises. Is that included intentionally, or has filtering code just not been written yet?

Can't extract questions

I can't decode the hash in RStudio. The decoding logic and ui elements work when deployed in the tutorial. I'm using the developer version of both learnr (learnr_0.10.1.9015) and learnrhash (learnrhash_0.2.0).

I (students) take the hash and paste it into a .csv file.
help.csv

data <- read.csv("gradebook\help.csv")
names(data)[names(data) == 'Answer.1'] <- 'hash'
soc294_questions <- extract_questions(data, hash)

But, I get an error like this:

Error: Assigned data map(data[[col]], vec_to_wide, col = col, names_sep = names_sep) must be compatible with existing data.
x Existing data has 2 rows.
x Assigned data has 5 rows.
i Only vectors of size 1 are recycled.

Feature request: extract_questions should have a 'correct' column

The extract_exercises function includes a boolean/logical column showing the correctness of the submitted answer (code), while the extract_questions function does not - it reports back the student's submitted answers. It would be great if the answer correctness appeared as well, so that tabulating scores could be simplified.

Encoder UI Text

Thank you for this package! It is exactly what I've been looking for in order to record students' interactions with tutorials.

I saw that one of the to-dos for the encoder UI is to allow the text and link to be dynamic. Is there an estimate for when this could be done? I am planning on releasing two tutorials for testing sometime soon, and it would be really helpful if the text could be changed.

Option to save answers in into rds file directly?

Have you considered providing an option which would allow users to save their answers directly into an rds file? This avoids the complexity of dealing with hashes directly. We are implemented such an approach here:

https://github.com/PPBDS/primer.tutorials

With the key code (modified from learnrhash) looks like:

submission_server <- function(input, output) {
  p = parent.frame()
  check_server_context(p)

  # Evaluate in parent frame to get input, output, and session
  
  local({
    encoded_txt = shiny::eventReactive(
      input$hash_generate,
      {
        objs = learnr:::get_all_state_objects(session)
        objs = learnr:::submissions_from_state_objects(objs)

        encode_obj(objs)
      }
    )

    output$downloadData <- shiny::downloadHandler(
      filename = "tutorial_responses.rds",
      content = function(file) {
        responses <- encoded_txt()
        readr::write_rds(responses, file)
      }
    )

  }, envir = p)
}

This worked well for us this semester. Students just uploaded the file directly into Canvas. But, first, we have no interest in maintaining our own version and, second, we can't figure out how to have the output filename be something different for each tutorial.

Sure hope that learnhash (or even learnr itself) provides this functionality directly someday.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.