otacke / h5p-essay

experimental H5P content for automated feedback on texts

H5P Essay

Try to give automated feedback

This is rather an experimental playground than a fully elaborated content type for H5P. In its first stage, learners will be able to write a text and receive feedback immediately. Initially (and maybe finally), this feedback will simply be based on a list of keywords that have been defined be a teaching person before.

The content type could possibly evolve into a solution for automated essay scoring using machine learning techniques. However, that's merely wishful thinking right now. Also: Beware the pedagogical implications and stumbling blocks!

Example

Done

• As a student, I can enter an essay text and submit it for "grading".
• As a student, I can (automatically) save a draft and continue writing later.
• As a teacher, I can add an introductory text describing the task and hints.
• As a teacher, I can add keywords that the text will be scanned for.
• As a teacher, I can define minimum number of characters in order to force students to write a certain amount of text.
• As a teacher, I can define a maximum number of characters for the essay in order to stirr thinking about the task.
• As a teacher, I can toggle fuzzy comparison that will grant some grammatical flexibility, e.g. also accept the plural form of a word.
• As a teacher, I can override individual settings for caseSensitive and forgiveMistakes.
• As a teacher, I can set individuals points for a keyword and its variations and thus weigh them for scoring.
• As a teacher, I can set a maximum number of occurrences of keywords and its variations that shall be awarded with points.
• As a teacher, I can disable scoring and just use autimated feedback in order to prevent wii-ficiation.
• As a teacher, I can decide that some keywords are mandatory while others are not. (more transparent via score only)
• As a teacher, I can define multiple ranges that will allow more diverse feedback.
• As a teacher, I can define optional feedback phrases for a keyword; one for "if found in text" and one for "if not found in text". The content type might glue those phrases together, thus creating a feedback text.
• As a student, I can receive instant feedback beyond a simple score.
• As a teacher, I can use regular expressions for keywords if I feel the need to.
• As a teacher, I can decide if the learner will see the keyword, an alternative, the matched word or none with the feedback.
• As a teacher, I can set points that will be awarded in the host system (e.g. LMS) for merely answering an unscored question.

Further Ideas

You're welcome to contribute your ideas as an issue on github. For now there are (in no particular order or guarantee):

User Stories

• As a teacher, I can receive the texts and the results of students.
• As a teacher, I can set different levels of difficulty that the students can choose from.
• As a student, I can select different levels of difficulty in order to gradually test myself.

xAPI statements

Compare https://web.archive.org/web/20170628002610/http://xapi.vocab.pub/datasets/adl/

• experienced (when opened): DONE (rather for activities without a "goal")
• completed (when submitted): DONE (but might be redundant)
• failed (if completed, but score is not sufficient): DONE
• passed (if completed and score is sufficient): DONE
• mastered (if completed and full score): DONE
• scored (if completed, submit numerical score): DONE

Misc

• text statistics and metrics, e.g. readability metrics or maybe something related to the AFINN file
• different levels of difficulty, choosable by teacher and/or student
• add css styling to feedback text (?)
• improve accessibility
• later on: possibly build a model that allows to relate texts with scores from a human, so you can train the content type and then use this model to compute the/a score, too.

About this repository

If you want to download the sourcecode, you can choose from three main branches:

• release: Will contain the latest official release.
• stable: Will contain features that have not yet been released, but that should work. Use at your own risk in a production environment.
• master: Will contain the latest progress, but may not have been fully tested. Should definitely not be used in a production environment!

h5p-essay's Issues

H5P-results from Essay not saved within Moodle

Hi,

Recently, the activity Essay was launched within H5P. Great! I also found a beta version of Dictation. Even greater!

But: the results that I score within both activities are not passed on to the Moodle grade list. That's less great.

Am I doing something wrong? Do I have to wait for an update of the H5P plug-in? Who knows the right answer?

Greeting, Kees Koopman.

Long evaluation time of essays if you enter lots of phrases / sentences

The evaluation time of a pretty short text can be astonishingly long, if the content creator has specified lots of sentences as expected answers. If the text becomes longer, the time would probably increase even more. So, some optimization of the detection algorithm might be good. I can provide an example of a content piece that is rather slow despite the fact that the expected text is so short.

Add Grammar Checker

Maybe there's a open Grammer Checker that can optionally be integrated.

Testing for Question Set

Test the implementation for Question Set thoroughly!

Add a thesaurus aaS

Add a thesaurus as a service, e.g. via the API of openthesaurus.de for German. Will need a general interface in order to offer the same feature for other languages.

Take care of xAPI "attempted" via setActivityStarted()

Grades are not reported to LMS?

Hello,

I tried out Essay and it doesn't report any grades to Moodle, is this a bug or a missing feature?

Fix a11y

Text should not have a tab index. It will be read by the screen reader when ordered to.
Change of focus should be announced.

Add capability for the teacher to view submitted essays

This is mentioned in the readme already, but I think this is a high priority feature, so I repost it here to make it more visible:

It would be really useful if teachers could view the submitted essays in their LMS. So in Moodle the gradebook module would display the texts the student entered. There are several H5P module that already have this capability (e.g. Quiz with Multiple Choice).

Catch empty fields

H5P allows to define mandatory fields, but doesn't enforce them being set. This can lead to keywords created accidentally that account to the full score. This should be catched.

essay.js fails to handle null previousState

When contentData is passed in with previousState == null, javascript error results ("Uncaught TypeError: can't convert null to object" at line 126 of essay.js in version 1.5.3) and content fails to display.

Description for "Forgive minor mistakes"

Describe to the user when a mistake is considered to be minor (in a reasonable way). Problem here: This might change in the future.

Check semantics for the "Labels"

Using a label for a library doesn't result in a label; not an issue with Essay, but possibly with the core.

buttons on top of progress bar

There seems to be an issue with the buttons' CSS or something similiar. If the horizontal space becomes too small, they are put on top of the progress bar.

Case sensitive switch does not work for RegExp

I have a keyword/phrase defined in RexExp format with the case sensitive option deselected. Ostensibly, the option is only valid for non-RegExp keyword types, since the option has no effect, whether on or off, if RegExp is used. It would be helpful if the option could simply apply the /i RegExp switch to the entry, since there is no way to include the switch in the keyword space. Alternatively, the RegExp switches could be accommodated somewhere else. Having /i or /x available really helps simplify some use cases.

Explain scoring

As suggested at https://h5p.org/comment/21179#comment-21179

It might be helpful to have some kind of optional legend that explains how the answer was scored.

Downside: It would push Essay even further into the realm of behaviorist teaching. Maybe I should not have included scoring at all but restrict Essay to verbal feedback.

IE 11 doesn't know append()

Object doesn't support property or method 'append'
e.g. solution.append(solutionTitle);

Bug

I am experiencing some weird behavior in an example I'm creating for H5P Essay. If you enter the correct solution in the attached exercise, the answer is evaluated as 11.2/11.2 even though all keywords are set to count "1 point". There are also actually 14 keywords.

Version of H5P Essay: 1.0.1

H5P-File (in a zip-file, as GitHub doesn't accept .h5p):

der-ausbruch-des-1-weltkriegs.zip

expected solution:

Nachdem der österreichische Thronfolger Franz Ferdinand von einem serbischen Separatisten am 28. Juni 1914 in Sarajewo ermordet worden waren, suchte Österreich-Ungarn die Unterstützung des deutschen Kaisers Wilhelm II., der seinem Bündnispartner im sog. "Blankoscheck" bedingungslose Hilfe zusagte. Daraufhin stellten Österreicher ein Ultimatum an Serbien, bestehend aus scharfen Forderungen. Obwohl Serbien darauf einging, erklärte Österreich ihm den Krieg. Daraufhin machte mobilisierte Russland, die Schutzmacht Serbiens, seine Armee, was zu Mobilisierungen bei den übrigen europäischen Großmächten führte. Bald befanden sich das Deutsche Reich und Österreich-Ungarn mit Russland, Frankreich, und Großbritannien im Krieg.

Allow negative points for keywords

See suggestion

• Can we allow negative points for keyword scores without breaking anything?
• How to interpret the score bar?
• How to interpret passing/mastering percentage?
• Don't screw with scoring of existing content types!

Complete ARIA support

Add ARIA support to feedback

Better support for conjugations

In addition to fuzzy comparison, allow wildcards (at the beginning or the end of a keyword) that may be very useful for language learning.

Smartest way? Use the * in the text field? If we do that, we might as well think about replacing the GUI keyword alternative addition with a |

Retry Button not available

Hi Oliver,

The "Retry" button is not showing up if nothing is placed in the "keywords".

Issue was reported here: https://h5p.org/node/986323

Test content: https://h5p.org/node/988300

Improve wildcard

Limit the number of characters that the wildcard should represent

Instead of inventing our own notation, we could simply allow the use of regular expressions that will be detected if a string starts and ends with a slash symbol.

Improve reporting

Add custom reporting to h5p-php-reporting or enhance reporting by adding feedback to xAPI object description.

Export option for the student

Right now, the text is saved in the host environment. It might be useful for students to export their text. I think the priority is low though, because students can also simply copy & paste their texts if they want to.

feedback about keywords not really easy to understand

Hi

We use the Essay type of exercises in our web site, but we do not have main characters

The problem is, in the image, the missing word is "from", but is really hard to understand that from the current feedback. Maybe something more neutral would be apreciate

regards

Minimum number of characters

H5P.TextInputField might be changed to also support a minimum number of characters. If both (minimum and maximum) can be set, we could use the range widget instead of two number fields.

Table with feedback for specific keywords should display actually found alternative

It's excellent that you can specify feedback if a certain keyword was found or not. However, I'd like to propose a change about how the feedback is displayed: if the user enters a keyword from the alternatives list, the feedback table still displays the "main keyword". It would be better if the keyword in the table was the actually entered keyword. If the "main keyword" is displayed it might confuse the reader, as he/she didn't really enter it.

Improve accessibility

Think about adding an image on top

https://h5p.org/comment/17664#comment-17664

H5P Essay question

Essay question currently allows the teacher to specify how many characters the learner can use in their response to the essay question, but the teacher is capped at 255 characters.
It would be extremely useful if the teacher can specify the length of the feedback as we often feel limited by the this character cap.

Add support to H5P Columns

Using Essay in H5P Columns might add the opportunity to combine it with e.g. images or videos that should be described. Should be tested thoroughly.

A single `*` freezes the browser

As described in the hint:
Keyword or phrase to look for. Use an asterisk '*' as a wildcard for one or more characters. Use a slash '/' at the beginning and the end to use a regular expression.
I tried to use a single * to match all words (give full marks whenever there's input), and the browser is freezing. Currently I'm using ** or/.+/ as a fix.

I suspect the problematic code is located at
https://github.com/otacke/h5p-essay/blob/master/scripts/essay.js#L674
where needle is replaced to be ``, and then stuck in the loop on line 678.

Add bulk import or textfield as alternate way of keyword collection

Optionally hightlight words that have been detected in text

• Add boolean option to semantics
• Add logic to highlight on check (will require to deviate from text input)
• Add logic to unhighlight on uncheck

Add translation for German

Pass 'i' flag to regex if !caseSensitive

This may be some low-hanging fruit that will greatly simplify regular expressions. Rather than specify caseSensitive in the Regexp options, use the configuration settings. This would allow users to make use of the i flag when writing their expressions.

String.prototype.startsWith()

Hi: This is a really great plugin. We're using it on our Moodle site for training mental health service providers using case studies. Thank you for your work!

One issue I noticed when clicking the Check button on an Essay instance in IE 11 is that nothing happens. I think this is because (in part or perhaps totally) String.prototype.startsWith() isn't supported in IE11.

I see that there are polyfills for this here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/startsWith#Polyfill

I'm wondering if you'd consider adding that for backwards compatibility? Hopefully it's an easy fix. As much as I'd love to ignore IE11, a lot of the health care service sector where I'm from in Canada still uses it in their offices.

Thanks again,
Matt

How to upgrade h5p essay to the latest version of this repo in my Moodle?

In my Moodle, it's 1.2. So I don't know how to upgrade to 1.4 in this repo. Thank you very much!

Ideas

I've giving your idea some consideration and I think it's really interesting. I work as a secondary school teacher and automatically graded texts could be an excellent supplement to marking them myself, which I rarely come around to as often as I want.

Here are my thoughts:

It would be useful, if the teacher could also see the texts and the feedback given to the student by the content type. That way you could check a random sample to make sure students take their work seriously. I haven't figured out yet, whether the current H5P integration into e.g. Moodle allows the teacher to see the student's responses that are stored with xAPI or if they are only accessible to the student. If this is possible, it could be done that way.

I think grading language as a second category would also be a useful feature (I teach English). This could include things like:

• was the text written in a paragraph structure
• does it use conjunctions to connect paragraphs
• how elaborate is the word choice
• how complex are the sentences
• spell & grammar check

The content creator could define which of these criteria he want's to apply and could further specify options, like expected complexity of sentences, etc. If the student didn't do too well in one area, he could receive pre-defined tips on how to improve.

Of course, this grading language would have to rely heavily on external libraries, most of this is impossible to implement on your own.

Furthermore, I think you should design the content type in a way that's language agnostic, providing the content creator with a choice of the language he wants to use. Even without grading language, getting plural forms of keywords, etc. means applying linguistic rules. If you hard-code these rules to deeply into the content type, it will only be useful for grading English, or whatever language you use.

xAPI for Essay

Hi, on h5p.org it lists no xAPI coverage for Essay, but reading around on comments it seems that some gradebooks like moodle are getting graded data. I'm using wordpress and it would be great to know if Essay can generate xAPI statements on correct answers. Sorry if the answer is out there and I haven't been able to turn it up yet. Thanks!

Forgive minor mistakes setting

Hello,

Essay has setting "This will accept minor spelling mistakes (3-9 characters: 1 mistake, more than 9 characters: 2 mistakes)." is only english language supported or what languages are? How it does the checking in f.ex. Moodle H5P?

Add support to H5P Question Set

Requested in https://h5p.org/node/163001

Solutions of multiple instances on one screen collide

Using '\n' in the keyword doesn't behave as intended

https://h5p.org/node/1427613

I've been using H5P Essay to ask students for the output of a simple code block, but I've been struggling to get multiline input working correctly. I couldn't find a correct input when using \n for the question I linked above.

A correct answer should look like:

You got a B!
You got a C!
You got a D!

Using /You got a B!\nYou got a C!\nYou got a D!/ as the keyword should match it, but it doesn't seem to.

I'm temporarily using \s to match any whitespace, and that seems to work for newlines too but it also accepts some undesired answers, like:

You got a B! You got a C! You got a D!

I presume this is a limitation with newlines being handled as spaces, but do let me know if there's an alternative solution that I'm missing. Thanks!

Support "Reizwortgeschichten"

Think about how "Reizwortgeschichten" could be supported:

• Draw a random set of words that needa to be included
• Allow to customize the task description based on the words that were drawn

Add optional "sample solution"

It was nice if the teacher had the option to an a sample solution the student could have a look at. Will require a "show solution button" and some thinking about a good implementation.

Wildcard and regular expressions only work with letters, not numbers.

When attempting to use a serial number with random numbers at the end of a fixed string as a keyword, neither the wildcard * or regex / are compatible with numbers preceding or following a search term. For example, if I attempted to use MM71.098.B as a term, the value MM71.098.BCD would be valid, but MM71.098.B99 would not.