snap-cloud / lambda-evaluator

Cloud

An interactive, project sharing site for Snap!.

What is Snap! Cloud?

This section is TODO

• Snap! GH Repo
• Snap! homepage
• scratch site

Background

The initial design and development of this project was during Spring 2015 for Professor Fox's CS169, "Software Engineering" course at UC Berkeley. The original team members were:

• Alec Guertin
• Arjun Baokar
• Jason Wang
• Linda Lee
• Michael Ball
• Steven Campos

Setting Up

This (currently) project uses Rails 4.2.1 and Ruby 2.2.1, and aims to be always up to date!

The Heroku destination for the app is https://ucbsnap.herokuapp.com

1. CLONE WITH -r git clone -r [email protected]:snap-cloud/snap-cloud.git
2. You should have rvm or similar installed.
3. bin/setup
4. bundle install --without production
5. git remote add heroku [email protected]:ucbsnap.git

Dependencies

• nodeJS (>= v0.10) and npm
  • Used for bower
• Ruby 2.2.1
• imagemagick for testing profile and thumbnail management
• OS X -- if you have brew there will be an install script sometime...

Development Notes

[ TODO: Move this section to a Wiki?]

• GitHub Flow [link needed...]

• The awesome print gem

$ rails console
ap SomeObject

Heroku Deployment

To deploy to Heroku, you should simply need to do a git push heroku master. However, the build process is slightly more complex than a traditional Rails app.

Heroku Deployment uses a custom "build pack"

1. Install nodeJS
2. Post Node install, install bower
3. Then run the normal Ruby buildpack
4. Install with bundler
5. Serve using the Puma (multithreaded) web server

Pre-Reqs to Deployment:

1. Make sure you have a Postgres database added to the Heroku environment.
2. That's it?

License

Please see the file LICENESE for a full copy of the current license for this project (AGPL 3).

lambda-evaluator's Issues

Integrate Rails HTML / JS changes into repo

Consider removing the show / hide passing test cases functionality.

• Is it necessary?
• Lots of passing cases could distract from the 'failing' ones?
• Passing tests should be shown by default anyway... (Confidence boosting and clarity.)

Test Dependencies

We should have a way of communicating test dependencies. Something like "You should probably fix this before continuing on to other parts" or "This error is causing 3 other errors below"

Redesign Autograding API

We should make available 1-2 functions / objects. Not totally sure what the API should feel like, but some ideas...

SnapAutograder seems like a good one.
autograder = new SnapAutograder(world)
SnapAutograder.registerTests({} || function() {})

AutograderConfiguration should be an object or function that lets you configure:

• where to post results to
• any log transformations
• callbacks functions for various events, "onRevertSubmission", "onRunTests", "onShowFeedback"
• a debugging log level

Switch to using standard bootstrap classes for AG feedback

.success translates to all tests passing
.warning translates to semi-passing or changes mage
.danger translates to failure?

Do we want to show red for failures?
We should only have 3 different states, IMO.

• success, changes made, (semi-failing || all failing)
  OR
• success, (changes made || semi-failing), all failing

Or we could just have 2 states which is nice and simple.

Auto detecting Changes is VERY Slow in some cases...

....and things lock up.

Not sure why this is.... :/

Can we reduce the scope of when stuff is serialized / checked?
Can we improve the speed at all?

Code should be colored with the block category color

Function Logger

http://stackoverflow.com/questions/5033836/adding-console-log-to-every-function-automatically

Rename AGTest → RegisterAGTests + Cleanup

• This function should be called in initializeSnapAdditions
• Called with world as the first parameter....or maybe not. I don't know if it's necessary.
• Should return an object with test data (And a Feedback log?...Hmm?)

Feedback Needs a set of standard variables and messages

• $BLOCKNAME could be a variable that is dynamically replaced with either an image or the block text.
• standard text for I/O error messages. $IO_MISMATCH or something
• $BLOCK_INPUT $BLOCK_OUTPUT
  • maybe have a way to get positional arguments? [1] [2] ?

Investigate / Fix loading Snap! with a #present URL

Bind Escape Key to closing the autograder feedback

I thought this used to work? Maybe I broke something....

Integrate new UI from rails app

...probably....

Not sure what the best way to handle this would be.

Headless Chrome TODO / experiment

https://developers.google.com/web/updates/2017/04/headless-chrome?utm_source=frontendfocus&utm_medium=email��

See Section on Docker

Pontificate on using React

w/ React.RenderDOM some of this could work nicely as a re-write.

Simple IO tests need feedback when tests fail.

There should be an easy way to show improvement.

Have a separate log format of DB logging

No need to log large image files in the DB.

How would I handle this?

Confirm ARIA attrs for navbar + overlay

Both should mirror the bootstrap docs.

The overlay should be a strict modal

GS Autograder Output format

https://gradescope-autograders.readthedocs.io/en/latest/specs/#output-format

After logging we need a generic "text to block" mechanism

When logging data anything that is a text version of a Snap! object should be tagged as such. If these are JSON logs we can just have a property like "snap-block" "snap-list" "snap-sprite", or use data-* attributes in HTML formatted data.

When we render the log to students we should then search for those items and turn them into images.

Merge with beautjoy/autograder

Hover bug with the tips button

it removes the bg color on hover.

Move the serialization and diff analysis to a web worker

It could happen in the background, and then on changes, the existing analysis could be stopped so we aren't just wasting cycles.

We should / could also probably wait 0.5s - 1s before starting the analysis.

Output should display spaces

To make debugging more clear.

Upgrade to Bootstrap 4

Design A Configuration API

Related to #29

Mostly, I think it should be as self-contained as possible.

Import snap-autograder.js and then that's all you'd need.

let autograder = new Autograder(config)

autograder.runAllTests()

config = {
    callbacks: {}, // TODO
    previousSubmission: (??, ??) => { return snap_xml },
    bestSubmission: (??, ??) => { return snap_xml },
    logSubmission: (submissionLog, serializedXML) { return result? }
}

Migrate issues from lambda repo....

Re-think messages in the header bar.

• rethink the different states and messaging the autograder can be in.
• Make buttons be clickable but explain state in the popups?

Test cases can chose to be shown or not

By default a test should always be shown, however, it would be nice to have a test case that's hidden until some event happens. Perhaps there can be a showWhen or shownIf function that is called for each test case.

What should such a function be called with? The log so far? Test results?

Remove dependence on global `id` variable

a task id should be something less common, and preferably in all caps or part of some other config format.

Need a function to normalize spaces

Every test should probably do this by default, unless specifically told not to.

Rethinking Display of Tests

Each Test should have a title -

After expanding the test we should show a more detailed representation of the test outputs, and include a diff of the output.

Question: How can we display a list morph in a decent way?

Show a message when tests are running

Rewrite native dom stuff in jQuery.

It's just easier to read...

Re-add the help content

Condense Repeat Errors

If errors are the same, we should try to condense them in a simplified way.