anubislms / anubis Goto Github PK
View Code? Open in Web Editor NEWDistributed LMS for automating Computing Science Courses From NYU
Home Page: https://about.anubis-lms.io
License: MIT License
Distributed LMS for automating Computing Science Courses From NYU
Home Page: https://about.anubis-lms.io
License: MIT License
https://github.com/os3224/os3224-assignment-4-almoss1 was names with '-1' appended to the repo name. All the /public/webhook parsing was totally broken.
https://github.com/GusSand/Anubis/blob/master/api/src/routes/public.py#L114
Please put these changes on the v3.0.4 branch
All of the blog components are here. There are two components BlogImg.jsx
and BlogPost.jsx
that are the standard components.
The BlogPost
is a component that is meant to be a simplified template component that posts can be put in. We just need to provide a Preview, and Post components (along with title, date, author). The Preview is what is shown on the /blog
page. The Post component is then what is the rest of the post that is placed under the Preview after the post is clicked on.
I converted the main [ElevatorPitchPost.jsx]https://github.com/GusSand/Anubis/blob/v3.0.4-CHG-open-source-prep/web/src/Components/Public/Blog/ElevatorPitchPost.jsx) post to fit in this format a few days ago.
I considered using react-markdown to simplify the writing of the posts, but I could not quickly find a way to control the preview of the post. If you can see a way to control what the preview is on the /blog
page before the post is clicked on, go ahead and do that.
One thing to note here is that the images that start with
/api/public/static/
will not load unless you have the local api running, and it has a copy of the static files that exist on prod.
From /api/public/auth/whoami
for me (I am a superuser):
{
"data": {
"context": {
"id": "...",
"name": "Intro to OS"
},
"status": null,
"user": {
"admin_for": [
{
"id": "...",
"name": "OOP"
},
{
"id": "...",
"name": "Intro to OS"
}
],
"github_username": "wabscale",
"id": "...",
"is_admin": true,
"is_superuser": true,
"name": "John Cunniff",
"netid": "jmc1283",
"professor_for": [],
"ta_for": [
{
"id": "...",
"name": "Intro to OS"
}
]
},
"variant": "warning"
},
"error": null,
"success": true
}
Right now we are using the normal mariadb bitnami chart on prod. It works fine, and supports master-slave replication. The problem with it is that it still has a single master node. This is quite annoying because whenever the main node is down, everything crashes.
We should migrate from vanilla mariadb to mariadb-galera which supports multi-leader mariadb.
It looks like we can mostly just replace the existing helm install with the new chart. I expect that some of the dns names may change. The new mariadb service may be galera.mariadb.svc.cluster.local
.
from difflib import SequenceMatcher
import json
import pandas as pd
def similar(a, b):
return SequenceMatcher(None, a, b).ratio()
a = json.load(open('./assignment-final-question-assignments.json'))
netids = list(a.keys())
first = next(a.values())
pools = list(map(lambda x: x['question']['pool'], first['questions']))
results = {_pool: [] for _pool in pools}
for netid in netids:
for question in a[netid]['questions']:
pool = question['pool']
text = question['response']['text']
row = []
for _netid in netids:
_text=None
for _q in a[netid]['questions']:
if _q['pool'] == pool:
_text = _q['response']['text']
break
ratio = similar(text, _text)
if len(text) == 0 or len(_text) == 0 or netid == _netid:
ratio = 0
row.append(ratio)
results[pool].append(row)
So I am trying to figure out how to implement the emailing thing. I cannot find an end point in api that is used to create an assignment. Do you guys just manually add assignments into the database? I also cannot find the CLI tool mentioned in the design doc.
I would like to add a contribution guide and some other actions workflow things like:
The design doc is currently rendered using pandoc. It requires haskell, and pandoc to be natively installed on the system. I would like to try to replace this with some kind of docker run command to remove this unnecessary dependency.
Another thing to consider is that the mermaid images in docs/mermaid are rendered natively as well. I believe we can put this in a container as well.
I would like to keep the cli that is packaged in the management IDE something that can be installed natively on a host machine. For linux power users like me, I will still prefer to use my own machine over the management IDE.
In the management IDEs, the anubis cli config is provisioned with an authentication token. https://github.com/GusSand/Anubis/blob/bb69516b243aeddcd870e8c95df46a93169a8fed/api/anubis/utils/k8s/theia.py#L95
# If this is an admin IDE session, then we should add a token
# for the anubis cli to be able to authenticate to the anubis
# api.
if credentials:
# Create a token for the session owner
token = create_token(theia_session.owner.netid)
# Create the INCLUSTER environment variable as the base64
# encoded token. This token should be picked up by the
# theia init process to initialize the anubis cli with
# the token.
extra_env.append(client.V1EnvVar(
name='INCLUSTER',
value=base64.b64encode(token.encode()).decode(),
))
The main thing here is building out some authentication mechanism for the cli. To get this to work on native, we'll need to add some way of authenticating, then setting this token in the cli.
The current readme is a 30 page design doc that probably should be in a separate directory. A new readme should be created that focuses on more helpful information like sections on:
After a github classroom link is clicked, there is a 5 to 10 second delay between when the repo is created and the webhook is sent to Anubis. The assignment page should be changed so that when the link is clicked, we poll the assignment listing for 60 or so seconds until it changes.
The benefit of doing this is that the Cloud IDE button will become available without needing to reload the page.
The high level assignment page state is here and the button is here
New features on the horizon for Anubis v3.0.0
There are a few places in the prod provisioning that assume that the do-block-storage
exists. We should replace these assumptions with something that can be specified on the command line. I would like to be able to run this from the k8s/prod makefile:
make all STORAGE_CLASS=something-else
We will need to figure out how to configure theia to run on a different base url path. Something like anubis.osiris.services/ide/proxy
would probably work fine.
We would probably then need to configure the theia builds here and here. Then we would need to remove the THEIA_DOMAIN config, and replace it with something like DOMAIN
.
Then quite seperately we would need to change the theia proxy IngressRoute definition
We have HPAs defined in the v3.0.4 branch, but my worry is that they are not fine tuned enough to scale up the deployments when they are needed.
We will need to really take time to understand the way that the Horizontal Pod Autoscaling works to tune these deployments properly. I believe that we will also need to define Pod Disruption Budgets for the items that are getting HPAs.
We have a problem. Anubis takes a lot of resources to run on Mac and or Windows because those OSs do not support native containers. If we can get the anubis API and WEB to run in some super mindebug mode on bare metal, we would be helping devs that chose MacOS and Windows over glorious Linux.
I expect this has the possibility to be too difficult and time consuming. If it takes too much time, then we should skip.
The current base images for Anubis live in a private registry in digital ocean. To encourage collaboration, we should put them in a public repo on dockerhub. Alternatively we could host our own docker registry that is available to be pulled publicly.
The only images that need to remain private are the assignment test images (which exist in other repos anyway).
Ideally, the Marketing/Landing website should be separate from the Anubis web application.
Like:
It is unclear to me if we should use keep it simple with an smtp server, or go more complex with integrating the gmail api.
An alternative to scheduling notification rpc jobs is just adding a function to detect when notifications need to be sent out, and put it in a k8s CronJob that could be run every 5 minutes or so. We could then enqueue jobs to send out the notifications from there. This setup would be much more fault tolerant than scheduling rpc notification jobs.
Then the last thing is that I have not decided if I want to use the google gmail api, or just a simple smtp server setup. In Anubis version 0.0.1 back in January of 2020 there was no website. I sent out emails for every submission to students via a simple smtp server setup. We ended up getting rate limited by gmail in the like 3 hours before the deadline in that first assignment. i.e. emails were not being delivered to students. Then alternatively maintaining google api credentials is just annoying. I have done it in k8s before and could copy some code over, but it is just more annoying than something simple like an smtp server.
Some pages (specifically repos and profile) look like they are not automatically redirecting to the auth when clicked on in an anon session.
The current landing page could use some improvements. It was written by me, which means it is simple as it can be. I am not a frontend / web design person so I could use some more keen eyes on this one.
p.s. The nav on the left is cut off because I used the firefox "screenshot of full page" feature. It is not actually getting messed up.
We currently track all responses that students enter. We have no way of accessing anything but the latest from the admin panel. I would like to add some kind of components to the frontend admin panel to make previous question responses accessible.
I think this task is pretty straight forward. We would need to add a page for students to see lecture slide postings, and something in the admin panel for admins to upload them.
My thought is that a lecture slide posting can be the file/files with an optional description. We can also think about (maybe not for this card) a way to set a release time.
One last thing I would like this feature to have is some kind of a docx/doc file warning. Since docx files are absolutely irritating (why should i need to open libreoffice when i can open pdfs in firefox) we could have a optional checkbox for those files where we will convert them to pdfs after uploading.
Looks like a file accidentally made it in the repo here https://github.com/GusSand/Anubis/blob/master/theia/proxy/yarn-error.log.
It may be worth adding yarn-error.log
to the gitignore so contributors don't accidentally add these files.
There are several individual things in Anubis that have their own dependencies. I know that there are several places that are installing things that are not being used. We need to clean those places up.
We have a description field on every assignment that is not viewable anywhere. I am thinking maybe we can add another button to the assignment card, with a material-ui dialog component.
It may also make sense to put the description in a react-markdown component so that things like links and images can be interpreted.
I'm thinking that this can either be:
mailto:[email protected]
a component. we can add a template subject and body like mailto:[email protected]?subject=demo+plz&body=gib+demo+plz
There are some network policies that are defined in network-policy.yml. These policies are only on the places where users have some level of execution on the Anubis cluster (like the theia pods and autograde pipelines).
We need to add policies to govern other places like the web and api deployments.
Use this tool for visualizing and editing network policies https://editor.cilium.io/
Really all I would like to have is some very simple click counter for the blog posts to track engadgement
In order to expand to other classes, parts of Anubis will need to be re-written or overhauled. This is the list of the primary things that would need to change for Anubis's adoption to expand.
We can and should be more intelligent about how we handle the autograde calculations. We know they can only changed for a student when new submissions are processed. Knowing this, we can potentially cache the results for much longer (say like a week or two) and clear the cached results when the submissions is processed.
From the flask-caching docs
user_has_membership('demo', 'admin')
user_has_membership('demo', 'user')
cache.delete_memoized(user_has_membership, 'demo', 'user')
I'm not sure what exactly should go in the privacy agreement. A general template is probably fine. I imagine that there are reactjs libraries for those annoying popups that are on most sites now.
We'll need to replace the number field on the lecture notes with a posting time field. We'll also need to reflect this change on the admin lecture notes page and public lecture notes page.
Take the pieces from the readme that would be helpful to TAs learning how to use Anubis to the admin panel. If they can access it on the website, then they won't need to bother me as much.
Github webhooks are not being delivered something like 10% of the time. That's making it so we are sometimes missing submissions.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.