ls1intum / artemis

Artemis - Interactive Learning with Automated Feedback

Artemis: Interactive Learning with Individual Feedback

Artemis brings interactive learning to life with instant, individual feedback on programming exercises, quizzes, modeling tasks, and more. Offering customization for instructors and real-time collaboration for students, this platform bridges creativity and education. Embrace a new era of engaging, adaptive learning and artificial intelligence support with Artemis, where innovation meets inclusivity. Find out more on https://artemisapp.github.io

Main features

1. Programming exercises with version control, automatic individual feedback (and assessment) based on test cases and static code analysis (executed using continuous integration) for any programming language.
   • Instant: Students receive immediate and individual feedback on submissions. Instructors can customize feedback messages easily, hide feedback during the working time (e.g., with hidden tests)
   • Interactive: Instructors integrate interactive instructions based on tasks and UML diagrams directly into the dynamic problem statements. They can define hints for difficult exercise parts.
   • Independent: Instructors can customize programming exercises to support any programming language. To simplify the setup, Artemis includes sophisticated templates for the most common languages (e.g., Java, Python, C, Haskell, Kotlin, VHDL, Assembler, Swift, Ocaml, ...)
   • Local and online: Students can participate locally in their preferred IDE or in the online code editor (without any setup)
   • Manual reviews are possible directly in the online code editor or in Orion
   • Policies: Instructors can define submission policies (e.g., penalties after 10 attempts) to prevent that students try out all possibilities without thinking.
   • Grading: Instructors have many options to configure grading, analyze the results based on tests and static code analysis categories and re-evaluate the results
   • Secure: Test cases and student code run in Docker environments on build agents. Test frameworks such as Ares simplify the creation of structural and dynamic tests and prevent that students can cheat.
2. Integrated code lifecycle: Elevating the programming exercise experience, Artemis seamlessly integrates version control and continuous integration, creating a streamlined, cohesive development environment. Unlike external systems, this integrated approach minimizes maintenance overhead and is tailored to the specific needs of a learning platform, offering unparalleled flexibility. It simplifies the initial setup for administrators and ensures a smoother, integrated workflow for students and instructors alike.
3. Quiz exercises with multiple choice, drag and drop, and short answer questions
   • Modeling quizzes: Instructors can easily create drag and drop quizzes based on UML models
   • Different modes: Quizzes support a live mode (rated) during lectures, a practice mode for students to repeat the quiz as often as they want, and an exam mode (see below)
4. Modeling exercises based on the easy-to-use online modeling editor Apollon with semi-automatic assessment using machine learning concepts
   • Multiple diagram types: Artemis supports 7 UML diagram types (e.g. class diagrams) and 4 additional diagrams (e.g. flow charts)
   • Easy to use: Students can create models easily using drag and drop, they can explain the models using additional text.
   • Integrated feedback: Reviews can provide feedback and points directly next to the model elements.
5. Text exercises with manual, semi-automatic assessment based on supervised machine learning and natural language processing (NLP) using Athena
   • Integrated feedback: Reviews can provide feedback and points directly next to the text segments.
   • Language detection: Artemis detects the language of the submission and shows the word and character count.
6. File upload exercises allow full flexibility to instructors. Students can create any kind of file (e.g. PDF, PNG) and submit it to Artemis when they have completed their work. Artemis allows instructors and tutors to download the files and assess them manually based on structured grading criteria (see below in the section Assessment).
7. Exam mode: Instructors can create online exams with exercise variants, integrated plagiarism checks, test runs and student reviews. You can find more information on Exam mode student features and on Exam mode instructor features.
8. Grading: Instructors can configure grade keys for courses and exams to automatically calculate grades and display them to students. Grades can be easily exported as csv files to upload them into university systems (such as Campus online). Instructors can optionally define bonus configurations for final exams to improve student grades according to their grades from a midterm exam or course exercises.
9. Assessment: Artemis uses double-blind grading and structured grading criteria to improve consistency and fairness. It integrates an assessment training process (based on example submissions and example assessments defined by the instructor), has a grading leader board, and allows students to rate the assessments. Students can complain or ask for more feedback.
10. Communication: Instructors can post announcements. Students can ask questions, post comments, and react to other posts in channels or private chats. Tutors can filter unanswered questions.
11. Notifications: Artemis supports customizable web and email notifications. Users can enable and disable different notification types.
12. Team exercises: Instructors can configure team exercises with real time collaboration and dedicated tutors per team.
13. Lectures: Instructors can upload lecture slides, divide lectures into units, integrate video streams, lecture recordings, and exercises into lectures, and define competencies.
14. Integrated markdown editor: Markdown is used to format text content across the platform using an integrated markdown editor.
15. Plagiarism checks: Artemis integrates plagiarism checks for programming exercises (based on JPlag), text exercises, and modeling exercises in courses and exams. It allows notifying students about identified plagiarism. Students can review and comment on the allegation.
16. Learning analytics: Artemis integrated different statistics for students to compare themselves to the course average. It allows instructors to evaluate the average student performance based on exercises and competencies.
17. Adaptive learning: Artemis allows instructors and students to define and track competencies. Students can monitor their progress towards these goals, while instructors can provide tailored feedback. This approach integrates lectures and exercises under overarching learning objectives.
18. Tutorial groups: Artemis support the management of tutorial groups of a course. This includes planning the sessions, assigning responsible tutors, registering students and tracking the attendance.
19. Iris: Artemis integrates Iris, a chatbot that supports students and instructors with common questions and tasks.
20. Scalable: Artemis scales to multiple courses with thousands of students simultaneously using it. In fact, the largest course had 2,400 students. Administrators can easily scale Artemis with additional build agents in the continuous integration environment.
21. High user satisfaction: Artemis is easy to use, provides guided tutorials. Developers focus on usability, user experience, and performance.
22. Customizable: It supports multiple instructors, editors, and tutors per course and allows instructors to customize many course settings
23. Open-source: Free to use with a large community and many active maintainers.

Roadmap

The Artemis development team prioritizes the following issues in the future. We welcome feature requests from students, tutors, instructors, and administrators. We are happy to discuss any suggestions for improvements.

• Short term: Further improve the communication features with mobile apps for iOS and Android
• Short term: Improve the REST API of the server application
• Medium term: Add more learning analytics features while preserving data privacy
• Medium term: Improve the user experience, usability and navigation
• Medium term: Add automatic generation of hints for programming exercises
• Medium term: Add GenAI support for reviewing exercises for instructors
• Medium term: Add GenAI support for learning analytics
• Medium term: Add the possibility to use Iris for questions on all exercise types and lectures
• Long term: Explore the possibilities of microservices, Kubernetes based deployment, and micro frontends
• Long term: Integrated on online IDE (e.g. Eclipse Theia) into Artemis for enhanced user experience
• Long term: Allow students to take notes on lecture slides and support the automatic updates of lecture slides
• Long term: Develop an exchange platform for exercises

Setup, guides, and contributing

Development setup, coding, and design guidelines

• How to set up your local development environment
• Server coding and design guidelines
• Client coding and design guidelines
• Code Review Guidelines

Documentation

The Artemis documentation is available here. You can find a guide on how to write documentation.

Server setup

Setting up Artemis in your development environment or a demo production environment is really easy following the instructions on https://docs.artemis.cit.tum.de/dev/setup. When you want to support programming exercises, we recommend using the Integrated Code Lifecycle.

Artemis can also be set up in conjunction with external tools for version control and continuous integration:

1. GitLab and Jenkins
2. GitLab and GitLab CI (experimental)

Artemis uses these external tools for user management and the configuration of programming exercises.

Administration setup

If needed, you can configure self service user registration.

Contributing

Please read the guide on how to contribute to Artemis.

Once your PR is ready to merge, notify the responsible feature maintainer on Slack:

Maintainers

The following members of the project management team are responsible for specific feature areas in Artemis. Contact them if you have questions or if you want to develop new features in this area.

Feature / Aspect	Maintainer
Programming exercises	@krusche
Integrated code lifecycle	@krusche
Quiz exercises	@FelixTJDietrich
Modeling exercises (+ Apollon)	@krusche
Text exercises	@maximiliansoelch
File upload exercises	@maximiliansoelch
Exam mode	@krusche
Grading	@maximiliansoelch
Assessment	@maximiliansoelch
Communication	@rabeatwork
Notifications	@rabeatwork
Team Exercises	@krusche
Lectures	@maximiliansoelch @bassner
Integrated Markdown Editor	@maximiliansoelch @bassner
Plagiarism checks	@MarkusPaulsen
Learning analytics	@bassner
Adaptive learning	@bassner @maximiliananzinger
Learning paths	@maximiliananzinger
Tutorial Groups	@FelixTJDietrich
Iris	@bassner
Scalability	@mtze
Usability	@rabeatwork
Performance	@rabeatwork
Infrastructure	@mtze
Development process	@FelixTJDietrich
Mobile apps (iOS + Android)	@krusche @maximiliansoelch

Building for production

To build and optimize the Artemis application for production, run:

./gradlew -Pprod -Pwar clean bootWar

This will create a Artemis-.war file in the folder build/libs. The build command compiles the TypeScript into JavaScript files, concatenates and minifies the created files (including HTML and CSS files). It will also modify index.html so it references these new files. To ensure everything worked, run the following command to start the application on your local computer:

java -jar build/libs/*.war --spring.profiles.active=dev,localci,localvc,artemis,scheduling,buildagent,core,local

(You might need to copy a yml file into the folder build/libs before, also see development setup)

Then navigate to http://localhost:8080 in your browser.

Refer to Using JHipster in production for more details.

The following command can automate the deployment to a server. The example shows the deployment to the main Artemis test server (which runs a virtual machine):

./artemis-server-cli deploy [email protected] -w build/libs/Artemis-7.4.3.war

Architecture

The following diagram shows the top level design of Artemis which is decomposed into an application client (running as Angular web app in the browser) and an application server (based on Spring Boot). For programming exercises, the application server connects to a version control system (VCS) and a continuous integration system (CIS). Authentication is handled by an external user management system (UMS).

While Artemis includes generic adapters to these three external systems with a defined protocol that can be instantiated to connect to any VCS, CIS, or UMS, it also provides 3 concrete implementations for these adapters to connect to.

Server architecture

The following simplified UML component diagram exemplary shows more details of the Artemis application server architecture and its REST interfaces to the application client.

Deployment

The following UML deployment diagram shows a typical deployment of Artemis application server and application client. Student, Instructor, and Teaching Assistant (TA) computers are all equipped equally with the Artemis application client being displayed in the browser.

The Continuous Integration Server typically delegates the build jobs to local build agents within the university infrastructure or to remote build agents, e.g., hosted in the Amazon Cloud (AWS).

Data model

The Artemis application server uses the following (simplified) data model in the MySQL database (notice that the actual data model is more complex by now). It supports multiple courses with multiple exercises. Each student in the participating student group can participate in the exercise by clicking the Start Exercise button. Then a repository and a build plan for the student (User) will be created and configured. The initialization state helps to track the progress of this complex operation and allows recovering from errors. A student can submit multiple solutions by committing and pushing the source code changes to a given example code into the version control system or using the user interface. The continuous integration server automatically tests each submission and notifies the Artemis application server when a new result exists. In addition, teaching assistants can assess student solutions and "manually" create results.

Please note that the actual database model is more complex. The UML class diagram above omits some details for readability (e.g., lectures, student questions, exercise details, static code analysis, quiz questions, exam sessions, submission subclasses, etc.)

Artemis Community

There is a growing community of university instructors who are using Artemis.

Communication

We communicate using GitHub issues and pull requests. Additionally, you can join us on Slack to ask questions and get support. If you are interested, please send an email to Stephan Krusche.

Universities / Schools with Artemis in Use

The following universities are actively using Artemis or are currently evaluating Artemis.

• Technical University of Munich
  https://artemis.cit.tum.de
  Main contact person: Stephan Krusche

• LFU Innsbruck, Uni Salzburg, JKU Linz, AAU Klagenfurt, TU Wien
  https://artemis.codeability.uibk.ac.at
  codeAbility project
  Main contact person: Michael Breu

• University of Stuttgart
  https://artemis.sqa.ddnss.org
  Main contact person: Steffen Becker

• Universität Passau
  https://artemis.fim.uni-passau.de (only accessible via the university network/VPN)
  Main contact person: Benedikt Fein

• Karlsruhe Institute of Technology
  https://artemis.praktomat.cs.kit.edu
  Main contact person: Dominik Fuchß

• Hochschule München
  https://artemis.cs.hm.edu
  Main contact person: Michael Eggers

• Technische Universität Dresden
  Main contact person: Andreas Domanowski

• Hochschule Heilbronn
  Main contact person: Jörg Winckler

• Maria-Theresia-Gymnasium München
  Main contact person: Valentin Herrmann

Interested universities

• HU Berlin
  Main contact person: Lars Grunske

• Westsächsische Hochschule Zwickau
  Main contact person: Heiko Baum

• Technische Universität Chemnitz
  Main contact person: Danny Kowerko

• Universität zu Köln
  Main contact person: Andreas Vogelsang

• Technische Universität Dortmund
  Main contact person: Falk Howar

• Universität Bielefeld
  Main contact person: Daniel Merkle

• Universität Ulm
  Main contact person: Matthias Tichy

• Imperial College London
  Main contact person: Robert Chatley

• University of South Australia
  Main contact person: Srecko Joksimovic

artemis's Issues

Export Quiz Statistics

To analyze the quiz participation, it would be nice to have an option to download which student selected which answer option in the quiz, e.g. using the following json structure

{
"Questions": [
{
"Question": "This is a Question",
"Answers": [
{
"Text": "This is answer a",
"Correct": true
},
{
"Text": "This is answer b",
"Correct": true
}
]
},
{
"Question": "This is another Question",
"Answers": [
{
"Text": "This is answer a",
"Correct": false
},
{
"Text": "This is answer b",
"Correct": true
}
]
}
],
"Students": [
{
"LRZ": "ab12cde",
"CourseOfStudy": "ComputerScience",
"Answers": [
{
"SelectedItems": [
1,
2
]
},
{
"SelectedItems": [
1
]
}
]
},
{
"LRZ": "cd12efg",
"CourseOfStudy": "TUMBWL",
"Answers": [
{
"SelectedItems": [
6,
2,
3
]
},
{
"SelectedItems": []
}
]
},
{
"LRZ": "de34ged",
"CourseOfStudy": "TUMBWL",
"Answers": []
}
]
}

Make the "Report a bug" and "Request a Feature" buttons more prominent

Is your feature request related to a problem?

It's not visible right away if someone is having a problem.

Describe the solution you'd like

Either fixed on one of the sides or top of the editing window.

🐛 Text exercise: Stuck on finding new assessment

Stuck on finding new assessment

Environment

Chrome

Downloading PDF file gives you f.txt file 🐛

Describe the bug

Instead of a PDF file you will be offered by Google Chrome a txt file with the name "f"

To Reproduce

Steps to reproduce the behavior:

1. Go to Artemis POM Lectures
2. Click on Lecture 1 Attachment with the slides
3. Try to save file to disk
4. See error

Expected behavior

A PDF file with the correct name and the .pdf file extension

Environment

 - OS: [Win10 Home]
 - Browser [chrome]
 - Version [73]

Alert the creator of a quiz question if the (short) question title is too long.

The (short) question title seems to be limited to a certain length. However, this is not visible to the creator of a quiz question. Saving of the question will fail with a very unspecific error message.

The console shows the following:

• Failed to load resource: the server responded with a status of 500 ()
• Saving Quiz Failed! Please try again later.

Ideally, the question creator should be directly informed if the text field's data is longer than the allowed amount of characters before trying to save.

Create files

As a student, it should be possible to create new files within the Online Editor.

Support new exercise types

It would be nice if Artemis is able to support other exercises than coding, e.g. modeling or quizzes

Required changes:

1. Adapt the object model / database: Create sub classes for the Exercise class, e.g. programming exercise, modeling exercise, etc and customize the exercise types by that

2. Configure the exercise without build plan

3. Extend the flag "allowOnlineEditor" to use different types of online views / editors, e.g. a modeling canvas or a quiz interface

4. Define a storage for the exercise type
   4a) Quizzes: e.g. Markdown (think about if it makes sense to store in a repository)
   4b) Modeling exercise: Create a new version of the online editor and possibly reuse existing Angular components, see https://github.com/ls1intum/ArTEMiS/blob/master/src/main/webapp/app/editor/editor.html

5. Configure the view for instructor / teaching assistants who review the results and who insert feedback

Group and Sort Exercises in Course Dashboard

Allow users to group the exercises according to:

• Exercise Type
• Release Date
• Due Date (standard)

For Release Date and Due Date, it should be displayed on a weekly basis

Leave or hide courses

Is your feature request related to a problem?

There is no way of leaving a course.

Describe the solution you'd like

I would like to be able to leave a course as a student.

Show Exercises which are not assigned to a course

When you create a new exercise, and you do not assign it to a course it is no longer editable nor visible in the overview.

Allow download of build artifacts

While conducting an exercise about Build and Release Management, students should be able to download the build artifact (e.g. JAR).

Make Quiz Exercises Test run for TAs more visible when exercise is not released

Currently, tutors can see quiz exercises, but they cannot test them if they are not yet released. To get a feeling for the quiz and to be able to (easily) check the quiz setup and content, it would be good if tutors would be able to participate in quizzes before they are released, similar to the coding exercises.

Show score in %

As a student, my current score should be clearly visible within the Online Editor.

Setup Programming Exercise directly on ArTEMiS

• Create a Project in VCS with 3 repos: exercise, test, solution
• Create a Project in CI with 2 build plans: BASE, SOLUTION
• Configure access rights for both systems
• Create example test cases and the correct pom.xml file in the test repo
• Create an Eclipse project, gitignore, README example, one Main.java file in the correct package path (including src) in the template repo
• Create an Eclipse project, gitignore, README example, one Main.java file in the correct package path (including src) in the solution repo

The following variables are shown in the user interface when the instructor creates the exercise:

• Exercise Name/Title
• Exercise ShortName
• Course ShortName (should be added to Course)
• Programming Language (default Java)
• Package Path (for Java, e.g. "de.tum.in.www1")
• optional: Main Class name
• optional boolean: includeStructuralTestFramwork (it always includes one example of a behavior test)

All required files are stored within ArTEMiS' src --> config folder so that this feature can be deployed self-contained

Support reset of online code editor in case of merge conflicts

Currently, Artemis detects merge conflicts and shows an error message to the student, but cannot handle them properly, e.g. in case the online editor version contains changes that conflict with new commits in the remote repository.

One solution would be that the student gets the chance to reset the online editor version to the remote repository (or even better, but more complex: the student can choose which version should be kept, either the online editor version or the remote version)

Entry point to this issue would be https://github.com/ls1intum/ArTEMiS/blob/master/src/main/java/de/tum/in/www1/exerciseapp/service/GitService.java --> Method pull --> insert a try catch to handle this case properly

Split courses by semester🚀

During my structured evaluation interviews for the tutors related dashboards, I collected a feedback from a tutor who said that it would be cool to have the courses split by semesters, to see in which semester some course have been taken.

I think #319 is a necessary prerequisite for this

Access old courses🚀

During my structured evaluation interviews for the tutors related dashboards, I collected a feedback from a tutor who asked to be able to access old courses, as a student, also after a courses is ended. They say material shared during the course is useful, and would be cool to be able to go back and review what have been done after a course is ended

Label exercises with categories

For the calculation of bonus scores with different kind of exercises, it would be helpful to allow instructors to tag exercises with keywords.
These keywords are then used to filter for certain exercises when calculating the score of students

Allow TAs to download certain repos as zip files in the instructor exercise dashboard

Add a new button with a popup. In the popup, TAs can enter student identifiers in a comma separated list. When they click on Download, a new GET REST call on the server (comparable to ExerciseResource and ExerciseService --> Archive) checks if the students with these identifiers have participated in the exercise, clones all repos, zips them in a file and returns this as input stream so that the file can be downloaded in the angular client.

✅ Extend Protractor End-to-End Tests for Quiz Exercises

Is your request related to a problem?

We want to prevent regression errors in ArTEMiS.

Describe the solution you'd like

We should add e2e tests for

• creation, and
• participation

of Quiz Exercises.

Additional context

We introduced Protractor End-to-End Tests in #140.
Right now, they cover Course creation and deletion, as well as login.

Text exercise: Visual indication for feedback; colours are not optimal

Is your feature request related to a problem?

A red or yellow colour might confuse the student as it looks "wrong", but the colour is just used to indicate the relationship between the assessment and the student's answer.

Describe the solution you'd like

Display the feedback when clicking on the answer (similar to the "notes" of Moodle that are attached to the PDF / image).

Describe alternatives you've considered

Use lines that show up on hovering and connects the answer with the tutor's feedback.

Show max points for exercises in the course dashboard

If max points is specified for an exercise, it should be shown in the course dashboard next to the exercise

Overview Participations for a specific course

As instructors, we want to have the possibility to see/analyze the data for students for a specific course. For example, I want to easily check the overall participations of student A for the course B which had in total 16 exercises so far.
I want to know, in how many exercises the students participated (14 out of 16).

Set email address of commits by ArTEMiS

Otherwise it defaults to the name of the current linux user, which is currently "montag".

InternalServer Error when editing a Course

When editing a course the overlay shows an InternalServerError. Nonetheless, the system seems to work as expected. See attached Screenshot:

Present Sample Solution

As Instructor, I want to be able to provide a sample solution for an exercise.
As Student, I want to see a sample solution of an exercise after the exercise deadline has passed.

Change "Creation" Message after starting a UML exercise

Support new statistics

Create bar charts how many students solved the exercise correctly and how many failed for a specific exercise

Create bar charts for the whole course

"Clone repository" modal does not disappear

... when clicking anywhere else.

Participation Details broken

Steps to reproduce:

• Select an Exercise
• Select the Participations

Expected: Timestamps etc. are shown
Actual: Only template placeholders are shown

Add existing question

To reuse existing questions (from other courses), we need a popup with a search and filter mechanism to select an existing quiz questions.

Properties popover auto closes when auto-save triggers

It seems that the auto-save closes the properties popover (when editing a class or association).

Environment

 - OS: macOS Mojave 10.14.3
 - Browser: Chrome 73.0.3683.86
 - Version: Artemis 2.8.0

Collapse or hide courses

The following feature has been suggested by a student:

it would be very neat to make it possible to collapse courses in the „courses overview“. For instance, in my screen, EIST and other „old“ lectures are shown but not relevant to me now. Maybe a filter option what should be displayed would be even better.

Save attribute / method after dismissing with click outside

Is your feature request related to a problem?

E.g. when I add an attribute to a class, I click outside the popover and expect the application to save. Currently the application just drops my input.

Describe the solution you'd like

Clicking outside the popover should add the property.

Describe alternatives you've considered

Add a button next to the input so I can click on it to save.

Emtpy TextInput after Deletion of attribute/method

Add continuous code quality monitoring

Hey, great to see how development continues on this project!

I recently stumbled upon Codacy, a tool for analyzing and monitoring code quality. Besides highlighting code quality issues, it also gives explanations why certain things are an issue, so it is also a great chance for all contributors to improve their skills.

How about trying it out on ArTEMiS? It's free for open source projects.

Cheers,
Dominik

Clone Repository: Hide & copy password

As a student, it should be possible to hide the sown password and copy it to the clipboard.

Show build log if compiling fails

Currently, if compiling fails, ArTEMiS only shows "No tests founds (Check for compile errors)". There should be a possibility to get the detailed error message from the Bamboo build log.

Crash & Error Reporting

To get more insights about crashes and errors, we should integrate error reporting tools.

Specify the difficulty level of exercises

Instructors should be able to specify the difficulty level "easy", "medium" and "hard" for exercises. This should then be shown to students in the course dashboard next to the exercise

Can't delete course

A course cannot be deleted because of sql constraint violations.
If a course is deleted ArTEMiS (hibernate to be more precise) tries to set the foreign key exercise.course to null, but null values are forbidden.

Supporting deleting courses may require a on delete cascade sql constraint rule (and not on delete set null). However, that means that not only the course is deleted, but also all exercises, participations and results (git repo's and bamboo build plans might have to be deleted manually; on delete cascade only clears the database)

Stacktrace:

org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:259)
	at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:225)
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:521)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:485)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:291)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
	at de.tum.in.www1.exerciseapp.service.CourseService$$EnhancerBySpringCGLIB$$32e41271.delete(<generated>)
	at de.tum.in.www1.exerciseapp.web.rest.CourseResource.deleteCourse(CourseResource.java:130)
	at de.tum.in.www1.exerciseapp.web.rest.CourseResource$$FastClassBySpringCGLIB$$1cc4836a.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
	at de.tum.in.www1.exerciseapp.aop.logging.LoggingAspect.logAround(LoggingAspect.java:51)
	at sun.reflect.GeneratedMethodAccessor323.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:620)
	at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:609)
	at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.aop.aspectj.AspectJAfterThrowingAdvice.invoke(AspectJAfterThrowingAdvice.java:59)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:68)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at com.ryantenney.metrics.spring.TimedMethodInterceptor.invoke(TimedMethodInterceptor.java:48)
	at com.ryantenney.metrics.spring.TimedMethodInterceptor.invoke(TimedMethodInterceptor.java:34)
	at com.ryantenney.metrics.spring.AbstractMetricMethodInterceptor.invoke(AbstractMetricMethodInterceptor.java:59)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
	at de.tum.in.www1.exerciseapp.web.rest.CourseResource$$EnhancerBySpringCGLIB$$15a0161a.deleteCourse(<generated>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:110)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:832)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:743)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:961)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:895)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:967)
	at org.springframework.web.servlet.FrameworkServlet.doDelete(FrameworkServlet.java:891)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:654)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:843)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at com.codahale.metrics.servlet.AbstractInstrumentedFilter.doFilter(AbstractInstrumentedFilter.java:104)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at com.hazelcast.web.WebFilter.doFilter(WebFilter.java:360)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.springframework.boot.actuate.autoconfigure.EndpointWebMvcAutoConfiguration$ApplicationContextHeaderFilter.doFilterInternal(EndpointWebMvcAutoConfiguration.java:261)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:115)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:112)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.rememberme.RememberMeAuthenticationFilter.doFilter(RememberMeAuthenticationFilter.java:158)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:169)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:134)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:206)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:121)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at de.tum.in.www1.exerciseapp.web.filter.CsrfCookieGeneratorFilter.doFilterInternal(CsrfCookieGeneratorFilter.java:34)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:124)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:66)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:106)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:87)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:121)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:522)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1095)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1502)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1458)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
	at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:72)
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:211)
	at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:62)
	at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3400)
	at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3630)
	at org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:114)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:465)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)
	at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:425)
	at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101)
	at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:177)
	at org.hibernate.jpa.internal.TransactionImpl.commit(TransactionImpl.java:77)
	at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:517)
	... 145 common frames omitted
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Cannot delete or update a parent row: a foreign key constraint fails (`exerciseapplication`.`exercise`, CONSTRAINT `fk_exercise_course_id` FOREIGN KEY (`course_id`) REFERENCES `course` (`id`))
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
	at com.mysql.jdbc.Util.getInstance(Util.java:387)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:932)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)
	at com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1274)
	at com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:780)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073)
	at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2009)
	at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5094)
	at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1994)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:208)
	... 159 common frames omitted

Participation Site unresponsive

If there are many participations, the corresponding site gets unresponsive, because all participations are loaded. Is there a quick fix that solves the problem? I could imagine the following solutions:

1. Paging: Only load a predefined number of participations (e.g. 20, 50, 100, maybe even configurable)
2. Immediately filter: before loading the results, the user has to filter the exercise
3. Data loading happens in the background and the site stays responsive

With respect to performance, I would prefer options 1 or 2.

Add Analytics

To track the usage of ArTEMiS it would be great to add Analytics/Tracking tools

Export / import questions

To make it easier to reuse quiz questions in future quizzes, an export / import mechanism into a structured file format would be useful

ArrayIndexOutBoundsException when invalid Bitbucket URL is given.

When an invalid URL (with less then 3 slashes in the path), is given when creating a programming exercise, an ArrayIndexOutBoundsException occurs.

🚀 undo

Is your feature request related to a problem?

I'm frustrated taht I cannot undo/redo my changes in the Modeling Editor

Describe the solution you'd like

An undo/redo function

Support team work

There are several issues that we need to address:

1. Access Management: Configure in Artemis which groups should be used from the user management service

2. Repository creation: As soon as one student starts the exercise, repository and build plan will be created. For other students of the same team / group, the exercise is already startet on the web interface of Artemis

3. Special case online editor: as soon as a student from a group opens the online exercise the first time, a new version of the same remote team repository is cloned to the Artemis server for this student --> important: then merge conflicts need to be handled properly and the student needs the possibility to choose which version should be used

Special case: If real-time collaboration should be supported: use websockets and only create one repository

4. Special case LTI: All students need to initialize the LTI connection. Artemis needs to report the score to all students of that group over LTI.

Required changes in the object model / database:

• Add a group object and connect it to a participation with that group --> use JHipster mechanism liquibase

Challenge: How can we identify how much a student contributed to the test success: for example each commit creates a new build: identify how many test cases are green after this build that have been red before.

Use a better date time picker for exercise management

The built in date time picker does not work on all browsers and has a very limited usability, we should switch to something like https://github.com/DanielYKPan/date-time-picker

Purge build plans on CI server

After the due date of an exercise, it would be good, if Artemis can delete all student build plans on the CI server without deleting data in the Artemis database so that statistics and results are still available. Another possibility would be to manually start this purge mechanism.

The repositories in the VCS don't need to be deleted, so that student could theoretically resume their exercise (in this case only the build plan would be recreated).

To allow for better statistics, Artemis should store the following CI date in its database:

• Results of each test case (this would require a change of the object model / database)
• maybe: build log (but this should not be interesting)

important: in case of build output files (e.g. jars in release management exercises), the build outputs would be lost, but in this case students could resume the exercise and create the build output again

Instructions panel can't be scrolled to the bottom

Describe the bug

The instructions panel of an exercise can't be scrolled until the bottom. Some parts of the class diagram can't be seen.

Environment

 - OS: macOS Mojave 10.14.3
 - Browser: Chrome 73.0.3683.86
 - Version: Artemis 3.0.0