nuprl / ocelot

An IDE for JavaScript, without the "bad parts".

Ocelot

A web-based IDE for JavaScript, without the "bad parts". See https://www.ocelot-ide.org for more information.

This is a README to get started hacking on Ocelot.

Dependencies and Cloud Configuration

1. Install Node and the Google Cloud SDK on your development machine. We use Node 14.

2. Create a project on Google Cloud Platform to host Ocelot. You can reuse an existing project, as long as you don't mind setting Cloud Datastore to Datastore mode permanently for that project. If you are already using Cloud Datastore in Native mode, you will need to create a new project.

3. A domain to host Ocelot. For example, you can use a subdomain of github.io, if you do not want to pay for a domain name,

4. Ensure your development machine is logged in to the Google Cloud Platform

   gcloud auth login
   gcloud auth application-default login

5. Create a project on Google Cloud Platform and note down its Project ID. If this is the only Google Cloud project that you are using, you can set it as the default project on your machine:

   gcloud config set project PROJECT_ID

   Otherwise, you will have to add the --project PROJECT_ID flag to all subsequent commands that manipulate Google Cloud resources. The remaining instructions assume that you set the default project.

6. Create a bucket for Ocelot to store files. For example, the following command creates a bucket called ocelot-ide-org-files, and sets the storage class to regional, which is a cheaper storage option than the default, which replicates data across multiple data centers. The -b on flag disables per-file ACLs and isn't strictly necessary.

   gsutil mb -b on -c Regional -l us-central1 gs://ocelot-ide-org-files

7. Create a bucket for Ocelot to store revision history. For example:

   gsutil mb -b on -c Regional -l us-central1 gs://ocelot-ide-org-history

8. Enable object versioning on the former bucket. Ocelot relies on object versioning to implement its history feature:

   gsutil versioning set on gs://ocelot-ide-org-history

9. Using your web browser, go to the Google Cloud Datastore console and create a database in Datastore Mode. (Do not use Native Mode.) You will need to chose the datastore location. We recommend choosing the same region that you used for the two buckets that you created above.

10. Using your web browser, configure the OAuth consent screen for the Google Cloud project. It is straightforward to setup the application for testing, which limits you to 100 users over the lifetime of the testing period.

11. Using your web browser, go to the Credentials page for the Google Cloud project. Click the Create Credentials button and then click OAuth client ID. On the next page, select Web application as the application type and enter the domain name at which you plan to host Ocelot.

12. Using your web browser, Enable the Google Cloud Build API and the Stackdriver Error Reporting API

13. Copy the file /env.yaml to /backend/env.yaml and edit it as directed in the file.

Building and Deploying the Ocelot backend

cd backend
npm install
npm run-script build
npm run-script deploy

Building and Deploying the Ocelot frontend

Edit the file frontend/src/secrets.ts.

Run the following commands:

cd frontend
npm install
npm run-script build

At this point, you need to deploy the contents of frontend/build to a static web hosting provider. For example, if you plan to use GitHub Pages, you can create a repository with the contents of frontend/build, push that repository to a new GitHub pages, and serve it.

Copyright

Copyright 2018--2020 University of Massachusetts Amherst

Copyright 2019--2021 University of Texas at Austin

Copyright 2020--2021 Northeastern University

Old Instructions Below

To setup and run backend locally

cd Ocelot/backend
yarn install
yarn run build && yarn run serve

Development Instructions

Occasionally, you will want to hack on ElementaryJS (EJS), and will want to test out your changes before deploying them. To do so:

1. Clone EJS: https://github.com/plasma-umass/elementaryJS/
2. Install dependencies of EJS and build it (from within the EJS dir):

   yarn install
   yarn build
   yarn test
   

3. Hack on EJS code, and re-build.
4. Tell yarn to make the local version of EJS available to other projects (still within the EJS dir):

   yarn link
   

   Yarn should then tell you:

   yarn link v1.10.1
   success Registered "@stopify/elementary-js".
   info You can now run `yarn link "@stopify/elementary-js"` in the projects where you want to use this package and it will be used instead.
   Done in 0.03s.
   

5. Change to the Ocelot frontend dir, delete any cached EJS libs, and tell yarn to use the local version of EJS:

   rm -rf node_modules/@stopify/elementary-js
   yarn link "@stopify/elementary-js"
   

   Yarn should tell you:

   yarn link v1.10.1
   success Using linked package for "@stopify/elementary-js".
   Done in 0.18s.
   

6. Compile the ocelot frontend, and deploy it locally:

   yarn build && yarn serve-local
   

7. Further changes to EJS will not require you to run the "link" steps again. Yarn remembers that you told it to use the local versions. Just recompile both EJS and the Ocelot frontend.

Once you are done hacking on the local EJS changes:

1. Push a PR to EJS with the updates and the version number updated in package.json for EJS.
2. Run npm publish to update the published version of EJS
3. Create an Ocelot PR to use the newly deployed version of EJS by editing the EJS version number in package.json in Ocelot/frontend
4. To tell yarn to go back to the published version of EJS: In the Ocelot/frontend directory:

yarn unlink "@stopify/elementary-js"
yarn install

Issues you may have

• Delete this directory: rm -rf Ocelot/node_modules

• There is a bug in the latest Yarn (1.7) that gives a stupid error when you use yarn install or yarn add after running yarn link:

  yarnpkg/yarn#5876

  So, first run:

  yarn unlink stopify && yarn unlink elementary-js && yarn link stopify-continuations
  

  Then run yarn install or yarn add ...:

  then in Ocelot/frontend, run:

  yarn link stopify && yarn link elementary-js && yarn link stopify-continuations
  

ocelot's Issues

Spreading array hangs IDE

This issue is related to nuprl/ElementaryJS#200

Problem

While ElementaryJS detects this as an error, Ocelot hangs due to the error being uncaught.

Step to reproduce

1. Input the below javascript code:

[...[1, 2]].length

Expected behavior

No error

Actual behavior

Ocelot IDE hangs. In developer console, it prints out error as below:

Slow Internet

Slow internet when logging in leads to files not properly loading in. I encountered this once when having slow internet. I'll try to reproduce it with chrome dev tools and see what is wrong.

Update the backend to not use the ocelot-settings repository

Using `yield` hangs IDE

Version

{
  elementaryJS: "c122f37",
  ocelot: "6887ff2"
}

Problem

When explicitly using yield in a generator function, ElementaryJS detects this as SyntaxError (SyntaxError: Unexpected strict mode reserved word). However, Ocelot hangs due to the error being uncaught.

Interestingly, using yield in an anonymous generator function works fine, and ElementaryJS doesn't detect as error.

Step to reproduce

1. Input the below javascript code:

function* generator() {
    yield 1;
    yield 2;
}
const range = generator();

Expected behavior

Print error in Ocelot console

Actual behavior

Ocelot IDE hangs. In developer console, it prints out error as below:

Unnecessarily file updates when not logged in.

Signing in and closing out of the google log in unnecessarily causes a file reset.
It would reset the code and files to its default state.

Remove hardcoded URL

Update https://github.com/umass-compsci220/ocelot-settings/blob/master/src/lib220.ts#L269 to not have a hardcoded URL.

GitHub Actions to build Ocelot

Formatting issue when used in a half-sized window.

When using the IDE in a half-sized window, the first line of text is overlapped by the banner. This can be seen in the picture below.

Support for sudo

Backspacing/Typing in editor

Holding backspace lags the editor. The reason is that backspacing continuously updates the editor thereby updating the state. The state changes cannot keep up with the editor. I thought of two solutions:
Right now the state is constantly changing when typing and the function to save code to cloud is debounced

1. We'll keep our debouced save to cloud function but rather than changing the state every time the code changes, we also debounce that as well. The only occasion when saving code to state and to cloud has to be done immediately is when user selects a different file to edit.
2. Redux state is based on immutability and so new states are created every time (It has to copy the old state from scratch and make changes on the duplicated version). Backspacing would not lag if the code changes are not done on the state but on an instance variable of the code editor component (this allows mutability which is a bit faster). It would first take in some initial code that is loaded from the cloud but all further code changes would take place on an instance variable.

Typing lags somewhat on slow computers (cheap ChromeOS laptop), having this fix would also benefit slow laptops.

Can refer to $S

Auto-Load Github Gists from URL query

When someone sends the browser to Ocelot with a link such as

https://ocelot-ide.org?gist=https://gist.githubusercontent.com/joydeep-b/8956ab7ab21e36a0f4c12fa289f952a4/raw/9613aec9bf17dea01d8c16ab16996897ae789fed/bouncing-ball.js

, Ocelot should just download the gist, and create a new file with the contents, so the user can just hit "run".
This will be most useful for linking to examples from the public webpage, https://umass-compsci220.github.io/Ocelot/

No scrollbar for file list

File list is not scrollable. If the number of files exceeds the number visible, the entire list cannot be seen.

Bad error message: free variables

Free variables do not get locations reported. The problem is that we use a proxy to catch unbound identifiers, and don't have original source locations at that time.

Console input history and resize if needed

Make console input remember previous console inputs and resize the console input if necessary (like in Messenger, Shift + Enter to go to newline but not run the code)

auto-save when loading new file

It is possible for the buffer to become dirty when a new file is selected, but not yet loaded. These updates may be clobbered in a 1 second window. Simple solution is to lock the buffer when a file is being fixing.

Newly Edited/Created files when not logged in

Up for debate, what should files that are deliberately created/edited by the user become after logging in. Right now, it just removes all the files, replacing them with the files one has on the cloud.
I suggest it automatically saves those deliberately created/edited files to the cloud once they are logged in. This becomes really helpful with cold starts. The sign in button is gonna take 5 seconds to log in and the user won't be able to stand to wait that long to start prototyping something with the editor. Once they sign in, the file they edited remains and it doesn't get cleared out.
(I know some companies schedule cron jobs to prevent cold starts from becoming a problem, if cold starts isn't a problem, this wouldn't be a bug)

Ocelot, or Paws?

Decide on the name, and rename the project repo name to match.

Should console.log yield control?

Today I tried to run effectively this program:

let exit = false;
while (!exit) {
  if ('redbull64' === prompt('Enter password')) {
    exit = true;
  }
  else {
    console.log('Wrong password');
  }
}

If the user keeps entering the wrong password, this program does not reliably display "wrong password" on each iteration. The problem is that console.log involves DOM updates that only display when the turn ends. Neither console.log nor prompt end the turn, so the turn only ends when Stopify suspends the loop, which is nondeterministic.

So, we need either console.log or prompt to suspend and immediately resume execution. I propose we modify console.log, so that it always prints immediately. This will significantly slow down programs that display in a loop, but it will make them more responsive.

Text formatting for compilation text

Format text such as these differently, so they are visually distinct from console.log output and REPL interactions:

9/11/2018, 12:58:17 PM EDT 
Compiling... 
Compilation succesful. 
Starting program... 
Program terminated normally. 

Logging out while files are loading

When user logs out when files are being loaded, files still get loaded

Implement proper undo for editor

Current bug is editor undo 'stack' is cleared every time new code is loaded (i.e. user selects a new file). I want editor to preserve the undo stack for each individual file. The solution is to implement 'faux' Tabs (basically hidden file tabs, not really tabs but function as them behind the scenes).

Reference: Monaco Editor Issue 604

ocelot-ide.org is running the development build of React

I spotted this while using the "React Developer Tools" plugin. Is this something worth mentioning because I think the production build enhances the performance of the page?

context in which the console evaluates is wrong

The console is broken for the following reason:

you can enter "window" and it shows the window object. The only variables that should be in scope are the variables in the user's program. I don't think this is a UI issue. This requires changes to Stopify.

Dirty changes result in full-page redraw

The code editor's onChange callback triggers a dirty flag to the code, which in turn triggers a redraw of the sideDrawer, thus resulting in keypresses taking 225ms for each key.

The fix: Don't disable the sideDrawer, just check when navigating away if dirty.

Minor Enhancement: Make login button pretty

Notify user properly if file is duplicated/invalid

Creating files with non '.js' ending leads to 'Duplicated file name'
The message should be 'Does not end in .js'

Failures of the is-regexp library result in spectacular crashes

https://console.cloud.google.com/errors/CJfe1s-Vv9-01AE?project=arjunguha-research-group

Stack trace:

Error: Symbol(Symbol.toStringTag) is not defined
    at Object.get (main.bundle.js:263)
    at Proxy.toString (<anonymous>)
    at ./node_modules/is-regexp/index.js.module.exports.input (main.bundle.js:136037)
    at stringify (main.bundle.js:316505)
    at ret.{.tokens.newLine.objKeys.map (main.bundle.js:316541)
    at Array.map (<anonymous>)
    at stringify (main.bundle.js:316536)
    at ./src/stringifyObject.ts.module.exports (main.bundle.js:316557)
    at OutputPanel.log (main.bundle.js:314361)
    at Sandbox.onResult (main.bundle.js:315972)
    at Object.get (main.bundle.js:263)
    at Proxy.toString (<anonymous>)
    at ./node_modules/is-regexp/index.js.module.exports.input (main.bundle.js:136037)
    at stringify (main.bundle.js:316505)
    at ret.{.tokens.newLine.objKeys.map (main.bundle.js:316541)
    at Array.map (<anonymous>)
    at stringify (main.bundle.js:316536)
    at ./src/stringifyObject.ts.module.exports (main.bundle.js:316557)
    at OutputPanel.log (main.bundle.js:314361)
    at Sandbox.onResult (main.bundle.js:315972)
    at main.bundle.js:159077

Highlight matching braces and parens

Can we do this? It would help. I don't want paren autocomplete however.

Move secrets to gitignored files

Move all client secrets outside of codebase and into a separate JSON file.

Here are two examples of client-related secrets that need to be moved: Login component and API Helper

Once it's all moved to a JSON file, I assume there's a need for a script that needs to be run before building the entire application (since plain JavaScript cannot read in local JSON files). The script would take whatever is in the JSON file, then create a new JS/TS file that certain files that need the client secret will import.

(Feel free to not use this idea if you have another idea in mind)

We'll most likely then reset all secrets, allowing project to be open sourced.

Misuse of "this"

This is an ElementaryJS issue, but we should fix it before release:

Dynamically check that this is not undefined in an object method.

loadImageFromURL is broken

This is broken on prod :(

Better acknowledgements of third-party code

I now remember that we have third-party code copied into this repository, for example:

https://github.com/umass-compsci220/Ocelot/pull/82/files#diff-2887097bfa77f555fb01358ac01e664dL3

We should do a better job of accumulating licenses and displaying them. Perhaps it can all go in the LICENSE file and the website can just have a link to the LICENSE on GitHub.

Console output creates links

The console hallucinates links, and makes things clickable. This should probably be reported to the console repos.
To repro, run this in the console:

let google = {com:0};
google.com

Output:

Suggested solutions:

1. If there is a setting, turn off the stupid "make hallucinated links clickable" setting for the console.
2. Workaround: style the links via CSS to make them appear as not links.

Client ID & Key configuration

Hi, I am trying to self host Ocelot, but it seems like I need to specify my own OAuth client key & id. I cannot seem to find the configuration to put them in. Where should I find it?

Bug in console-feed affects Ocelot

samdenty/console-feed#5

When the log is clipped, normal/error output gets mislabeled.

Let us not try to work around this and wait for a fix to console-feed.

Weird error message

This isn't valid JS, but it's a weird message to tell you that.

New file errors do not get cleared

Steps to reproduce:

Click on "New"
Enter an invalid name, e.g. "BAD"
Abort the new file creation, by clicking on the delete icon.
Click on "New" to start over.
Error behavior: At step 4, the new file entry shows the last error message.
Expected behavior: At step 4, the new file entry should not have an error message.

Fix anonymous mode

Dragging slider between code editor and canvas

Dragging the slider between the code editor and canvas all the way to right leads to the mouse being offset.
A gif to explain:

It would be nice if it does not do that.

Refactor backend's downloadURL method

Use these libraries:

https://www.npmjs.com/package/request
https://www.npmjs.com/package/request-promise-native

@joydeep-b This feature, including test cases, is a good starter project for anyone who wants to get involved in Ocelot hacking.

Inconsistent code view from same file open in different tabs

It's possible to accidentally lose changes with the same file open in different tabs / browsers / computers.
History lists the files correctly, but the diffs are incorrect.
Possible resolution: check if file is stale when tab gets focus?

History button should be disabled when no file is selected

Silently consume innocuous failures of GetURL CF

https://console.cloud.google.com/errors/CObptNjk5avmVw?time=P1D&project=arjunguha-research-group
https://console.cloud.google.com/errors/CPLa7KHE4vy8fA?time=P1D&project=arjunguha-research-group
https://console.cloud.google.com/errors/CLzn3cjJjo-CIQ?time=P1D&project=arjunguha-research-group
https://console.cloud.google.com/errors/CO7y3K-vh4vTKw?time=P1D&project=arjunguha-research-group

Another case in point for exceptions being bad.

Fewer files

There are 71 TypeScript files in the frontend and only 4,200 lines of code.

Some sort of reorganization is necessary.

Ability to login with an access code

This is something that we will need over the summer: we will give students a set of access codes to login. after login, the system will behave exactly as it does right now. We cannot assume that all students will have access to a Google Account.

I suggest we build a new start panel that shows before the IDE, with three options:
1. login with a google account
2. login with an access code
3. use without logging in

I think this will simplify the design of JumboContent, which has a lot of cruft to manage the transition between logging in and being logged out.

I suggest we create a CLI tool to manage access codes.

Default canvas to hidden on site load

Bad error messages (classes)

Spectacularly bad error if you invoke a class without new:

TypeError is not defined at : in _classCallCheck

The solution is to fork the babel class transform.

Bad error message

This is really a Stopify issue:

function foo () {
}

function bar () {
foo().bar(0);
}

bar ()

The error is app6 is undefined

Console does not show old copies of objects

When objects are displayed in the console, they are not snapshots of when they were printed. If you modify the object, even OLD console views of it update when redrawn (collapse & un-collapse)