expertiza / expertiza

Expertiza is a web application through which students can submit and peer-review learning objects (articles, code, web sites, etc). The Expertiza project is supported by the National Science Foundation.

Expertiza

Peer review system

Expertiza is a web application where students can submit and peer-review learning objects (articles, code, web sites, etc). It is used in select courses at NC State and by professors at several other colleges and universities.

Setup

NCSU VCL image

The expertiza environment is already set up in NC State's VCL image "[CSC517, S18] Ruby on Rails / Expertiza".

Using the VCL is the quickest way to get started, but you may find it awkward developing on a remote machine with network lag and having to reinstall gems every time you connect. Installing locally can be a pain though too. Life is full of tradeoffs. :-) The good news is that you can start on one environment, push your work to git, and switch to another environment if you don't like the one you started with.

Steps after getting a VCL reservation

• Clone the code
• Run bash setup.sh
• Run sudo gem install bundler -v 1.16.6
• Run bundle install
• Open config/database.yml and add the password as expertiza
• Run rails db:migrate
• Run rails s

Installing locally

See the Google doc on setting up the Expertiza development environment.

_{Depreciation warning: See the Expertiza wiki for setup instructions. Please update the wiki with corrections or additional helpful information. (http://wiki.expertiza.ncsu.edu/index.php/Development:Setup:OSX, http://wiki.expertiza.ncsu.edu/index.php/Development:Setup:Linux:RHEL, http://wiki.expertiza.ncsu.edu/index.php/Creating_a_Linux_Development_Environment_for_Expertiza_-_Installation_Guide)}

Contributing

• Fork the expertiza project
• Create a new branch for your contribution with a descriptive name
• Commit and push until you are happy with your contribution - follow the style guidelines below
• Make sure to add tests for it; the tests should fail before your contribution/fix and pass afterward
• Send a pull request to have your code reviewed for merging back into Expertiza

Style Guidelines

We've had many contributors in the past who have used a wide variety of ruby coding styles. It's a mess, and we're trying to unify it.

All new files/contributions should:

• Use unix line endings (Windows users: configure git to use autocrlf)
• Indent with 2 spaces (no tabs; configure your editor) both in ruby and erb
• Follow the Ruby Style Guide style for syntax, formatting, and naming
• Follow the design guidelines for the views.

When editing existing files:

• Keep the existing tabbing (use tabs instead of spaces in files that already use tabs everywhere; otherwise use spaces)
• Keep the existing line ending style (dos/unix)
• Follow the Ruby style Guide on code you add or edit, as above

Please do no go crazy changing old code to match these guidelines; it will just create lots of potential merge conflicts. Applying style guidelines to code you add and modify is good enough. :-)

Instructions to get production database ssh into a computer running Expertiza, e.g., an NCSU VCL node. ssh into expertiza.csc.ncsu.edu On production server, run mysqldump -uroot -p --databases expertiza_production > dump.sql #exports database from production On VCL, run sudo iptables -I INPUT -p TCP -s <IP_OF_PRODUCTION> -j ACCEPT #allows SCP requests Run SCP to your VCL scp dump.sql <unity_id>@<VCL_IP>:/home/<unity_id> mysql -uroot -p expertiza_development < dump.sql #loads database into production

expertiza's Issues

Email reminder thread may be causing 100% CPU usage

Occasionally the server takes up 100% CPU and comes to a crawl. It still processes requests, but very slowly. Both View and DB times are high.

List scroller for precedence constraints is too big!

For topics on signup sheets with staggered deadlines, there's a SelctionInList for choosing what topics must precede the topic one is currently looking at. (I.e., a list of topics that could have a precedence constraint wrt. the current topic.) This list is so large (about 12 lines) that it prevents the creator of the assignment from seeing more than one or two topics at a time. It should be changed to a dropdown, so that only one line is needed to display.

error evaluating nil.topic_id when viewing scores

Ed Re: Error in viewing scores

I am encountering a bug when I attempt to view scores of the current-semester assignments "Backchannel app" and "OSS project".

This may be because no reviews have been entered for the assignments yet. Other assignments that have reviews entered can do "View scores" correctly.

Here is the error that I am getting:

NoMethodError in Grades#view

Showing app/views/grades/_submitted_files.html.erb where line #4 raised:

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.topic_id
Extracted source (around line 4):

1: <% if participant.get_submitted_files() %>
2: <b>Submitted Files</b>   <a href="#" id="filesLink" name="filesLink" onClick="toggleElement('files','files');return false;">show files</a><br/>
3:   <div id="files" style="display:none">
4:      <%= display_directory_tree(participant, participant.get_submitted_files(), true) %>
5:   </div>
6: <% end %>
Trace of template inclusion: app/views/submitted_content/_main.html.erb, app/views/grades/_submissions.html.erb, app/views/grades/_participant.html.erb, app/views/grades/_team.html.erb, app/views/grades/_teams.html.erb, app/views/grades/view.html.erb

Expertiza is not time zone aware

Everything is currently in UTC

Team taking up a phantom 3rd slot

Ed: This is confirmed--OSS project E12--They took up one slot, plus a phantom third slot. Another team was also registered.

Can't delete assignment with a null submission directory

Read this thread from bottom to top; it describes what's wrong:

I tried, but it tells me "Assignment directory is not empty; delete anyway?" I say "yes", and get the message again.

Strange thing is, no assignment directory is specified, nor does one exist in /local/pg/pg_data/efg.

So something must be wrong with the check for an empty submission directory ...

Thanks,
Ed

Lakshmi Ramachandran [email protected] 12/13/2010 10:36 PM >>>
Yes, Dr.Gehringer.

Regards,
Lakshmi

On Mon, Dec 13, 2010 at 10:25 PM, Ed Gehringer [email protected] wrote:

Hi Lakshmi,
I keep seeing the "email reminder" assignment at the top of my assignment list in Expertiza. Can I delete it?
Thanks,
Ed

YAML error should be resolved by preloading model classes

Earlier I solved a problem where session[:credentials] was a YAML object by reloading the credentials every time this happened. It was a YAML object, because the model's class hadn't been loaded yet after a reboot when YAML::load was called. Instead, we should preload the class so that YAML can recreate the objects properly.

Commits:
5287386
471df00

Hide finished assignments

The assignment list is very cluttered.
Hide old assignments that have been completed.

No emails are being received

250 uni00mo.unity.ncsu.edu Hello nom26763d.nomadic.ncsu.edu [152.14.240.155], pleased to meet you
MAIL FROM: [email protected]
550 5.0.0 [email protected]... 2010.1015 This address does not exist and is now
blocked.

Lakshmi:
I've tried that but it didn't work. I tried changing settings to contain gmail's settings and my account details, but that didn't work either. I am at present out of options that I could try. The console isn't displaying an error. It says message delivered and even displays the entire body of the message with header details (from, to, subject, bcc etc.).
Could it be something to do with the time at which the message is to be sent? Time.now is current time + 4 hours and I am not sure if that is when the message is set to be delivered? But I haven't received any delayed messages either (4 hours after running the server).

Invalid dates should not be allowed to be saved

The problems we had with the Wiki textbook 2 re-review can be traced to an invalid date format that I must've inadvertently typed when I changed dozens of deadlines, two for each topic. All of this trouble would've been avoided if there'd been a sanity check for the date that is being saved to the db. I think there is such a check from the Create Assignment page (I could be wrong, though). But evidently there is no such check when one changes a due date for a staggerd-due-date assignment topic.

There needs to be one.

Restrict impersonation based on roles

The impersonate feature should be changed so that it is not possible to impersonate someone with greater privileges. As it stands, TAs can impersonate instructors.

Also, TAs should not be able to impersonate TAs for other courses. Ditto for instructors

Create/edit assignments should use same code

Currently, it's possible to create assignments with an arbitrary number of rounds of review. However, if you edit an assignment, you can't change the number of rounds. Obviously this is a DRY problem, and should be resolved by using the same partial for creating & editing assignments.

Problem importing metareviewers from csv

Xuezhou Ma [email protected] Re: Metareviews impossible too

Also, does assigning meta-review follows the EXACT same format as assigning regular review (shown in Chapter set 3.xls)? I worked out the meta-reviewer list, but it gave me some error when I tried to import the .csv file. I believe it is a format thing.

The excel generated csv (comma based): http://tinyurl.com/27c4a4u

The error shown is below:
Error!

The following errors were encountered during import.
Other records may have been added. A second submission will not duplicate these records.
Contributor, S30, was not found.
Contributor, S30, was not found.
Contributor, S30, was not found.
Contributor, 3a, was not found.
Contributor, 3a, was not found.
Contributor, 3a, was not found.
Contributor, 3b, was not found.
Contributor, 3b, was not found.
Contributor, 3c, was not found.
Contributor, 3d, was not found.
Contributor, 3d, was not found.
Contributor, 3d, was not found.
Contributor, 3e, was not found.
Contributor, 3e, was not found.
Contributor, 3f, was not found.
Contributor, 3f, was not found.
Contributor, 3f, was not found.
Contributor, 3g, was not found.
Contributor, 3g, was not found.
Contributor, 3h, was not found.
Contributor, 3h, was not found.
Contributor, 3h, was not found.
Contributor, 3i, was not found.
Contributor, 3i, was not found.
Contributor, 3i, was not found.
Contributor, 3j, was not found.
Contributor, 3j, was not found.
Contributor, 3j, was not found.
Contributor, , was not found.
Contributor, , was not found.
Contributor, , was not found.
Contributor, , was not found.
Contributor, , was not found.
Contributor, , was not found.
Contributor, , was not found.
Contributor, , was not found.
Contributor, , was not found.

time comparison with nil

time comparison with nil when displaying assignments that have duedates with nil due_at dates

Described in email subjects: [Expertiza crash?]; also [Blocking issue in Expertiza project][Expertiza error Project 8]

Migrations may refer to old models

082_add_new_questionnaire_types:
execute "UPDATE questionnaire_types set name = 'Review' where name in ('Review Rubric' or 'Rubric')"

Switching an assignment's rubric after reviews have started causes issues

If an instructor changes the rubric (questionnaire) for an assignment, the questionnaire gets updated properly. However, the Scores that were generated still point to questions on the old rubric.

In this state, some students cannot review (nil NoMethodError), and users reviewed under the old rubric receive a 0% grade.

Feedback submitted, but the author can't see it

Reported by Sameer Deekshit [email protected]:

After submitting: "Your response was successfully saved. Continue?"

I am not able to submit feedback for Reviews 1 and 2. I am able to submit feedback for Review 3, 4 and 5.

when I click on Continue, it redirects me to the your scores page. However, the reviews are not saved. When I reopen the feedback, it again appears blank.

Ed: The feedback gets saved, but Sameer sees a blank feedback form when he reopens it.

File submission fails: error evaluating nil.review_mappings

When submitting a file to the Backchannel app: (user: chan3)

NoMethodError in SubmittedContentController#submit_file

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.review_mappings

First line of tab-delimited file not imported

Ed:
Seemingly, when one imports review mappings from a tab-delimited file, the first line is not imported. An example: http://tinyurl.com/2e8eg78

Should be a way to change to/from Student view

Most systems used for classwork allow instructors, admins., etc. to act as students. In order to do so, they must "Switch to student view." In Expertiza, admins can act as students too, but they do so by using different pull-down menus. This is potentially confusing, especially for Assignments, which appear both in the admin and student menus,

Error importing excel csv from mac; fails silently

From Jeff Pittges [email protected]:
I reported a problem last week when I was unable to import teams and reviewers. My files were created with Excel 2008 for Mac. I saved the files to my Windows PC this weekend and I was able to successfully import the files to create teams and reviewers. Adam worked on the original problem and identified a problem with line breaks. The code change that Adam made eliminated the system generated stack trace error message but now the import operation fails silently when using files created and submitted from a Mac.

Javascript error when adding a nonexistant username as a participant

When I try to add skrishn2 as a participant to Final exam demo times, the screen jumps after I type the first character, and when I submit, I get this message:

Error!

undefined method `url_for' for #

Now, it turns out that skrishn2 doesn't exist (it should be skrish2). However, there's clearly something wrong with the error that is being reported.

Thanks,
Ed

'show authors feedback' link might show the wrong reviews

rsjohns3 on wiki textbook (see email re: Spontaneously generated author feedback):
http://expertiza.ncsu.edu/grades/view_my_scores/9926

When I click on the 'show authors feedback' link, sometimes I see all the reviews made by reviewers on my own topic (!!), and sometimes I see reviews by students of the review comments I posted for their topics. I am forced to reload my IE7 web page to correctly get the 'show authors feedback' link to work correctly

Can't submit link for wiki assignment

The "submit a URL" box is supposed to be showing up here, but it is not.
http://tinyurl.com/23mzdvc

Stage status is not displayed

On the assignment list, add two columns to show the current stage and when the current stage is due.

Team registered for 2 topics, can't drop topic E02

Ed> There is a team, "Awesome!", consiting of mlsurrat, crdzoba, and djfreema, which is registered for two projects in the Final project assignment. They want to drop E02, but on all of their signup sheets, they show up as registered for E01.

Security issue

Fix the reported security issue

Due dates for topics shouldn't be recalculated

Every time one creates a new signup-sheet topic that specifies a prerequisite topic, ALL the due dates for all topics are recalculated, with those having no prerequisites given deadlines equal to the earliest deadlines for any topics. This should be fixed by never auto-moving due dates back in time. Due dates could be postponed, but only with the instructor's OK.

Re: Not able to submit link for wiki text

Missing teams for staggered deadline team assignments

On staggered deadline team assignments, there are frequently reviews that point to a team that no longer exists. Figure out why, and come up with a solution to make sure reviews always have a team to point to that includes the author(s).

"Error!" when assigning a reviewer to a team

I've been having a problem assigning cadkins11 to review the Fast Food Junkies team for the F10 Project Description assignment. When I try to add cadkins11 as a reviewer I get a banner at the top of the page that just says Error!. There is no error message. cadkins11 shows up as a reviewer, but when I impersonate cadkins11 and click on Other's Work the page says that metareviews cannot be done at this time. There is no link labeled "begin" to start his review.

When I go to the Add Participants page for the F10 Project Description assignment, cadkins11 is the last participant in the list.

I just tried adding cadkins11 as a reviewer again. I got the error this time, but when I impersonate cadkins11 the "begin" link is now available.

It appears that the students has what he needs, but someone should look into this error and provide a meaningful message describing the error.

Thank you,
Jeff Pittges

Capistrano deploy

Expertiza should deploy using capistrano

E-mail reviewers function does not work

Attempting to e-mail the reviewers of a certain team, from the View grades page for OSS project, I received this error:

NameError in Grades#conflict_notification

Showing app/views/grades/_review_table.html.erb where line 11 raised:

undefined local variable or method `prefix' for #<ActionView::Base:0xb2a6c900>
Extracted source (around line #11):

8:  <tr class="head"><td align="right"><b><%= rowlabel %></b></td>    
9:    <% for review in reviews %>
10:            <td align="center">                   
11:            <% if prefix %>     
12:              <a href="#<%= prefix+"_"+review.map.reviewer.name %>"><%= review.map.reviewer.fullname %></a>&nbsp;<%=link_to '<img src="/images/delete_icon.png"/>', :controller => 'response', :action => 'delete', :id => review.id, :return => 'instructor' %>                
13:        <% else %>
14:          <%= link_to review.map.reviewer.fullname, :controller=>ctrl, :action => caction, :id=> review.id%>

Bogosity in numbers of reviews completed and outstanding

On each page where reviews are listed, the message "You have completed x reviews, with y reviews outstanding" appears. This was added, I believe, when the dynamic reviewer-assignment code was integrated into production. In the case of statically assigned reviewers, these numbers are usually--if not always--wrong! They must be removed to avoid confusing reviewers.

error evaluating nil.questions when students click begin for round 4 wiki reviews

Xuezhou Ma [email protected] Re: Setting up reviewing of wiki set 3 topics

NoMethodError in ResponseController#new

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.questions
RAILS_ROOT: /local/pg

app/controllers/response_controller.rb:147:in `get_content'
app/controllers/response_controller.rb:89:in `new'
/usr/local/bin/mongrel_rails:16:in `load' 

Signup sheet drop date

There should be a date after which a person cannot drop a topic they signed up for.
This should be configurable and have a sane default.

List of Projects to Review should have title & date of last update beside them

This helps spot projects which haven't had any submission & more easily locate an individual project to tweak it's scoring.

Topic in assignment list is always blank

In the assignment list (/student_task/list), the Topic column is always "—". Participant#get_topic_string always returns this, because Participant#topic is always blank.

Perhaps we shouldn't be pulling this from Participant#topic

Can't upload files to a team assignment when you have no team

Himanshu Arora [email protected] 12/4/2010 12:34 AM >>>
Hello Sir,
I'm trying to submit final project files but expertiza is throwing some error after 90% of uploading. Would appreciate if you could let me know any other alternative, though I've checked in the code at svn locker. Whole archived expertiza file is about 4.5MB in size.

Error when trying to assign a reviewer to a team

From Jeff Piddges [email protected]:

My students are experiencing two problems with reviews. I received an email from Dr. Gehringer about ten minutes ago saying a review problem had been fixed, but I received the errors below moments ago when I impersonated two of my students to verify that the problems were fixed for my students.

The review deadline for the F10 Project Description assignment is set to 2010-10-29 23:35:00. I believe this means 11:35 pm tonight. It is now 11:27 pm. Several students have reported that they logged in this morning and they were able to do their review but when they logged in this afternoon they received the message below. This seems to indicate that the system enforced the deadline several hours too early.

Two of my students just confirmed that they are still seeing this error.
Reviews for F10 Project Description

F10 Project Description Review 1 Begin (Work has not yet been submitted)

Metareviews for F10 Project Description

The error associated with the second problem is shown below. I impersonated zmorris and when I clicked on the Other's Work link I received this error message. I just received a third error (see below).

NoMethodError in Student_review#list

Showing app/views/student_review/_responses.html.erb where line #12 raised:

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.user_id

Extracted source (around line #12):

9: <%team = TeamsUser.find_all_by_team_id(review_mapping.reviewee_id)%>
10: <%@Participant = Participant.find_by_user_id_and_parent_id(team.first.user_id,@assignment.id)%>
11: <%else%>
12: <%user_id = TeamsUser.find_all_by_team_id(map.reviewee_id).first.user_id %>
13: <%@Participant = Participant.find_by_user_id_and_parent_id(user_id,@assignment.id)%>
14: <%end%>
15: <%else%>
� <%end%>
15: <%else%>

"Error" box appears every time reviewers are assigned

Seems that whenever one assigns reviewers, a notifier box saying "Error!" pops up on the next screen. The action is successfully performed, though.

Reviews have no Begin link

There's no Begin link for some people, but it tells them they still have reviews outstanding to complete. Especially prevalent in Wiki textbook 2. crdzoba is one such user.

Email notification should provide a URL to the update

Current message:
One of the submissions you are reviewing for the "OSS project" assignment has just been entered or revised. You may log into Expertiza and review the work now.

A URL should be embedded to open the relevant page direct

Fuzzy time strikes again -- are some functions on EST and others on UTC?!

I, as well as some students working on CSC 517 projects, have noted that Expertiza seems to enforce deadlines a little late.

Specifically, this evening at 12:35 AM EST, my Scavenger Hunt code review assignment was shown as being in the review phase, but the "Others' work" links were still grayed. The submission phase was supposed to have ended at 4:55 AM UTC (11:55 PM EST). So reviewing should have been enabled.

I set the submission deadline back two days, and was able to see the review links.

What this indicates to me is that something is out of sync between the designation of the assignment as being in the Review phase and the graying of the "Others' work" link. I wonder if graying is running on EST, but the phase indication is running on UTC! If so, this would explain a lot about why deadlines don't seem to be enforced at the times they should.

Textarea size too small

It seems that somewhere along the line, all review feedback boxes were compressed to one line. This is undesirable, as it gives the impression we are only interested in one line of feedack.

I think that the "additional comments" blank should be several lines tall. The others should be two or three lines.

Thanks,
Ed

Availability of assignments

There should be a flag that controls whether assignments are available to students. If an assignment is available to students, it must have at least one due date. This flag should be a field in the assignments table.

Merge dynamic reviewer assignment functionality from projectA branch

This needs to happen before the end of November

Joining a team causes expertiza to lose everything you've done on your previous topic

While one cannot sign up for two topics in the same assignment, after doing one topic, one can join someone else's team. And if you do that, Expertiza "forgets" that you've done your previous topic. Your reviews on the first topic then need to be retrieved from your partner's display or from database queries! (Discuss w/Ed ideas on how to fix this.)

Merge Self Selected Reviewing project

This would be helpful to Jen

Missing pagination links when filtering users by letter

If there are more than one page full of users with IDs beginning with a certain letter, there used to be a set of links down at the bottom of the page to pages listing all the users with IDs beginning with that letter. It looked like "1 2 3 4 5", if there were 5 pages of users whose IDs began with that letter.

Since the 2.2.2 upgrade, these links are missing. It is possible now to see only the first 20 users whose IDs begin with a particular letter. Can the links be restored?

Thanks,
Ed

Copying assignment + changing submission directory causes submission directory to be renamed!

Yesterday, I copied my "Battleship code review" assignment and renamed the new copy "Scavenger Hunt code review". I changed the submission directory from [efg]/csc216/f10/battleship to [efg]/csc216/f10/scavenger_hunt. To my surprise, this RENAMED the battleship directory to scavenger_hunt, with the consequence that all the submissions went in the same directory with submissions from the other assignment!

I have resolved the problem by moving the files to the correct directories, but the bug remains: When an assignment is copied and the submission directory path is edited, it should result in a NEW path being associated with the copied assignment, and should NOT rename the directory, which has the effect of making two assignments share a submission directory. If the directory name is changed in a situation not pursuant to copying an assignment, it would be reasonable to ASK the user whether the name of the existing submission directory should be changed, or a new directory created.