netsec-ethz / scionlab Goto Github PK
View Code? Open in Web Editor NEWSCIONLab user interface and administration
Home Page: https://www.scionlab.org
License: Apache License 2.0
SCIONLab user interface and administration
Home Page: https://www.scionlab.org
License: Apache License 2.0
See https://github.com/netsec-ethz/scionlab/wiki/Data-migration
Depends on #78
Provide a way to perform automated end-to-end tests
Add an admin action to promote/demote an AS.
Create a Form: prompting for:
Create a View to create a new user AS using this form. On submit, this will:
Create a View to see the details of and edit an existing AS, using this form. On submit, this will:
Not part of this issue: trigger configuration update for attachment point
Add support for (privileged) users sending an invitation (by email) to potential new users. The email should contain a link to a pre-filled registration form.
We can hopefully make use of either of the following libraries:
Currently, when creating a UserAS
, the Host
running the corresponding border router interface on the attachment point is selected using the function AttachmentPoint.get_host_for_useras_interface
which simply makes a sensible choice.
If we have a setup with an attachment point AS running on many different hosts, we may have to make this configurable.
The end-to-end tests will be great to determine whether the generated configuration actually works, but it will still be useful to have tests that verify that the generated configuration is identical to a known baseline (for faster feedback during development and to narrow down the cause of broken end-to-end tests). Maybe, we can simply check-in entire config tar.gz files and compare against those.
Most templates have been created without styling, forms are rendered plainly using form.as_p
.
Try to use django-crispy-forms
for form rendering.
user_as_add
/user_as_details
: form already looks decent, consider using crispy to simplify the codeSetup the continuous integration system & provide guidelines for adding tests. This should cover
unittest
)Check that current configuration can be adequately represented in new data model. Determine the required steps to import the data into the new coordinator & create issues.
Infrastructure:
Coordinator:
All the TRCs and Certificates have an expiration timestamp. Currently, this is just set "far enough" in the future to avoid having to deal with this. With all the functionality to update and re-issue certificates we already have in place, it is should be easy to set a mechanism that updates certificates before they expire.
We should be able to use recurring tasks from huey for this (adding huey is already on the way).
Add an admin action to check the current network topology configuration for problems.
Note: related to #46; all the checks proposed there as validation can also be double-checked here.
Plan and validate and implement the solution.
The requirements are:
account_id
and account_secret
at least the first time they contact the new Coordinator.May require some model changes as well.
Add an API to obtain the full configuration for a host, as a tar.gz-ball, containing
gen/
folder for the attachment pointThe parameters for the API are
The response is
Add a script/binary (sh, python, go?) to run on the host, which will query this API, unpack the config tar-ball and restart services as required.
For user-ases, running this script may be used as alternative to manually downloading and unpacking the tar ball.
For "managed" Hosts, this update-config script is run either
Note: might be able to re-use the script created created in netsec-ethz/scion-coord#299
Add a functionality to the scionlab server to trigger this config deployment on managed hosts. Either using ansible or plain ssh.
This triggering should
Add a new function create_gen(host, directory)
For one Host
object, generate the configuration for the gen
-folder, by traversing all the related Interface
and Service
objects.
Always write a configuration for sciond
.
After every change to a user AS, the configuration of the related attachment points (two APs involved if the user changes the attachment point) needs to be updated.
Note: If required (e.g. due to VPN-IP-assignment issues) we can re-implement the "updates pending" status, to disallow further changes to the user AS before the deployment has been completed. For now, I'd like to be optimistic that we can solve these issues without such a status.
For the infrastructure ASes we can support using IPv4/6 by (manually) setting a per-Interface public_ip
. For the APs, we'd have to automatically choose which address to configure, based on whether the user configured a IPv6 address. Currently, there is no field in the model where the options for IPv4/6 addresses could be stored; a Host
has only one default public_ip
.
Add back support for image builder.
Low priority. Image builder might be entirely replaced by having binary packages.
In the admin pages, creating/changing links can create an invalid SCION network topology.
We should implement proper validation to avoid creating impossible configurations:
Other things that can be validated (less important):
Customize the ModelAdmin
classes for the scionlab-model objects, to quickly get a somewhat workable admin interface to configure the infrastructure ASes (i.e. a replacement for scion-web).
Add a function to generate the VPN client configuration for a Host
: write_vpn_client_config(host)
.
Add a function to generate the VPN server configuration for a ManagedHost
: write_vpn_server_config(host)
Note: this should support multiple VPN clients on the same host.
Import Users from scion-coord data dump.
See https://github.com/netsec-ethz/scionlab/wiki/Data-migration
Depends on/connected with #75
Use django-recaptcha to add a captcha to the registration view.
The integration of the huey for the config deployment tasks is fairly fiddly. We should have unittests that only test the actual fiddling with huey.
Somewhat tricky to set this up for testing because the huey consumer needs to run in a separate process.
Create the main page for a user: this contains
Gather status info from attachment points / core-ASes.
This should then enable to automatically disable UserASes that have not been active for some time.
Adapt the default user model to:
Adapt the django_registration.forms.RegistrationForm
and the corresponding template (depends #3).
See notes on extending django user model:
Configure PostgreSQL instead of sqlite as the DB backend, at least in the production configuration.
Need to ensure that IP addresses given by user are safe to use in the attachment point configuration.
The UserAS creation/edit form needs a little bit of JavaScript logic for the following dynamic behavior:
Related: #31
The file under gen/overlay
is missing when generating the configuration. Without it, at least the current BS refuses to start.
Use the "two-phase" registration model from django-registration: https://django-registration.readthedocs.io/en/3.0/
django-registration
module as described here:INSTALLED_APPS
listCreate a nice landing page.
Add/populate navbar, hamburger menu with "my ASes", "logout" and "password change" and link to scion-architecture, tutorials etc.
Currently implemented by a simply delaying for one minute before actually running the update. All updates during this time will be bundled together. This introduces unnecessary delay.
As suggested by netsec-ethz/scion-coord#231
Implementation:
django_registration.forms.RegistrationFormTermsOfService
We should use a vagrant box with prebuilt SCION binaries, as in @juagargi's netsec-ethz/scion-coord#262.
As first step of configuration deployment, the AS certificates and TRC need to be generated if marked as needs_update
.
The models contain options to configure a bandwidth-test-server (Service.BW
) and pingpong-server (Service.PP
). During the generation of the gen/
folder, these are currently ignored in the code.
Add a supervisor config file to start these services.
A starting point is _create_gen
in generate.py:
scionlab/scionlab/util/generate.py
Lines 53 to 60 in 106fa96
Currently have (manually configured) servers that use UDP/TCP setup; two openvpn servers one UDP, one TCP, both handing using the same address range for VPN addresses, using an address-learn-script to explicitly set routing rules per client.
We'll need this, or something similar again to have "no UDP" fallback.
Create a script that creates the data model representation for the existing SCIONLab infrastructure topology (somewhat analogous to the current fixtures/testtopo.py
).
This should:
The idea is to start using the new coordinator to start configuring (some of) the scionlab infrastructure hosts.
The attachment points will not yet be included, but we could still create them as "dummy" entries.
(Let's not bother with the VPN settings yet).
Many home users have public IP addresses but those are dynamically changing (e.g. users connected over DSL or LTE). Those users could be supported by relying on a static hostname (such as one assigned through dyndns) instead of VPN, thereby avoiding performance degradation.
Profiling of the DB models and queries. Can be based on the existing tests. Preferrably also using PostgreSQL.
The goal is to detect big performance problems early and make the necessary adjustments to the data model.
Use django.contrib.auth
to implement login, logout, password change and reset functionality.
https://docs.djangoproject.com/en/2.1/topics/auth/default/#using-the-django-authentication-system
scion-coord hashes passwords using golang.org/x/crypto/scrypt
.
Django does not support this hash algorithm out of the box. We may be able to use django-scrypt, but it appears to be a bit outdated -- we might have to update or re-implement parts of it.
Add gen/ia
file, containing the ISD-AS (no trailing newline) to the config tarball, like the current coordinator does.
Not having this will break web-app and various other scripts/tools/apps that introspect the gen
folder.
Add a button to the AS detail view to download the configuration tarball.
This should just generate the tarball and then serve it. If there is a simple way to cache the tarball to disk using django's caching framework, then we should enable that.
Create script to compare generated configuration tar-ball with existing gen/
-folders.
When a user registers, she will currently just be redirected to login, without any indication that an email has been sent to activate the account.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.