ckrintz / appscale Goto Github PK

We have the license file in most of the projects, but not all of them yet.
Please verify which projects have it in here as you put them in.

Currently we start up new images using the cloud-tools with the --kernel
and --ramdisk args. In Euca 1.5 these args must be omitted to ensure
compatibility with KVM, and force the user to put these flags on -bundle-image.

Interfacing MySQL to AppScale.

The current SSH key name used in appscale, "ssh.key", is a name that may
interfere with other users' common choices for key names for EC2/Euca
instances. Should change this to "appscale" or "appscale.key" and verify it
works as intended.

Appscale-run-instances takes as args the kernel, ramdisk, and disk image to
start up, but since the average user only sees them once, they should be
read as environment variables to prevent the user from having to memorize
them or look them up every time they want to run their instances.

Once an image is powered off, all data on the instance is lost if EC2 or
Eucalyptus is used. Need to use Elastic Block Storage or S3 to save data
and retrieve it intelligently.

If two users are attempting to use AppScale and not enough nodes are free
for the second user, the tools fail to detect this and silently fail. Need
to change this and make sure enough virtual machines are free beforehand.

Currently running appscale-run-instances yields one master and three
slaves. Need to change this behavior to use the -min flag and have it
produce one master and (min-1) slaves.

Have done a quick review of the code involved and the changes involved
should be relatively minimal.

Change this later to be other sizes (ec2-run-instances instance-type)

Have the Djinn server set up firewalls for all ports which should be
private within the appscale machines.

When running appscale-run-instances, it runs ec2-run-instances. If that
command returns an error, we need to check for it and fail if it is an
error (otherwise the image will never come up and we shouldn't wait longer).

This will also require updates to hbase and hypertable.

What steps will reproduce the problem?
1. Start up appscale with an app which requires login
2. Try to login and reach the "everything looks good page" with Google 
Chrome
3. See that the redirect does not work and you're stuck on the same page

What is the expected output? What do you see instead?
Firefox and IE have the correct functionality. The page should be 
redirected to the application.

Use encryption for the appscale login and create account. Right now, all 
passwords are in the clear. Use a self signed cert.

HyperSoap saves the full table on stores, rather than just the individual
item. This can get slow if the table grows large.

Right now we wait 2 hours for the images to be started in Euca. Since Euca
enforces a 1 hour timeout by default then there's no point in waiting 2
hours, so change this to match.

Many parts in AppScale rely on it being the only thing in the cloud. Need
to change code to keep instance ids in mind when looking over ec2-tools output.

Known places where this is done:
  - vm_tools: just waits until a node has an ip address instead of waiting
for the node we fired up to have an ip. (same thing with helperfunctions on
AppController)
  - appscale-terminate-instances: just blindly kills all ec2 instances

Hardcoded the location of the keys in that databases stub in GAE. Need to
be able to pass in args to specify the location of the keys. 

The way we locate the app.yaml file in the tar'ed appengine app doesn't
correctly find it for Jaiku.

Change entity storage format to improve performance, and allow for
filtering at the data store level.

Need a new controller that can show where the apps in this instance are as
well as report on cpu / mem usage

Although the HTML/CSS shows the AppScale logo, the formatting on certain
pages prevents it from being displayed sometimes.

Must fix underlying libraries to properly untar and find app.yaml on cases
where appengine app is not in the same directory (gets confused by .. and ~)

Occasionally SOAP calls will fail with the following message:

Nirvana@bahamut:~$ appscale-upload-app -file httpmr.tar.gz 
We saw an unexpected error of the type OpenSSL::SSL::SSLError with the
following message:
SSL_read:: decryption failed or bad record mac.

Need to catch this exception and retry if this happens.

Clicking on the "logout" button does not actually destroy your cookie, and
thus does not actually log you out of the system.

Currently the Xen deployment option forces the key to be named
ssh.key.private and becomes a hassle to work on two Xen deployments at a
time. Working on using the keyname arg to allow it to work in a similar
manner as the Euca way.

-- Currently working on Xen, need to regression test against Euca before
committing back.

If a user chooses 'appscale' as the keyname in their cloud and another user
chooses the same name, the tools fail to work. Need to detect if the chosen
keyname is in use and fail if this is the case.

Integrate the newest version of AppEngine to get all their recent and most
up to date fixes.

Checksums need to be introduced to verify items in storage. The soap server
has checksums for the user and app table, but it is currently not being used.

Need to ensure ACID transactions. Currently does not implement rollback. 

Currently the tools only notice arguments with a single dash (e.g., -table)
when in fact if the argument is long then it should be double dashed (e.g.,
--table). The tools need to recognize both (for reverse compatibility) in
order to conform to UNIX standards better.

- Changed common_functions/parse_args to parse args in the requested manner
and changed the tools to accept them in the new format.

If an application is updated, fields may be added (and maybe deleted?).
Data from previous versions should be able to map over correctly. A version
number should be added to the application tables to distinguish between
versions.

The appscale-reset-pwd tool currently is non-functional. It normally calls
the PBServer's change_password method, but for an unknown reason it now
throws an RPCRouting exception. Will look over with Raj once integrating
the new directory layout is completed.

Currently once the head node is fired up all the user sees is a series of
dots until the PBServer comes up on the head node. The user has no
indication of how far the bootup process has come and if it fails, the user
never finds out.

- Current changes:
If the -v flag is not used, the behavior is the same as usual
Otherwise, the current state of the head node is reported every ten seconds
(not reported if the state is unchanged from the last time).

Need to finish integrating support for Cassandra into AppScale.

Databases should communicate encrypted, for security in a public cloud. One
concern is the communication between masters and slaves in Hadoop.

Need to change name to match what's in the tech report and keep an eye out
for the following:

- change any hardcoded paths to be relative
- change /etc/init.d/appscale to reflect new path
- change appscale-tools to reflect new path

Users and apps should be able to be disabled/re-enabled, although not fully
deleted.

Currently we lack a mechanism to automatically apply changes to the
AppScale code base and automatically fire it up. Right now we have to make
code changes, upload the new image to Euca, and manually fire it up (which
can take quite some time and requires the user to be on hand at all times).

Will be started once we work out the current bugs preventing version 1.0.4
from working.

Since our disk image works in Eucalyptus it theoretically should work in
EC2. However, we need to ensure it actually works in EC2 and grab numbers
about how much it costs to do so and get instructions on how a user would
do it.

Issue has been delayed to 1.0.3 release and will begin testing on 3/23/09.

Need to merge Raj's image with changed appengine layout with Chris' image
with changed appcontroller and apploadbalancer and test against Xen and
Eucalyptus.

Need to rename load balancer to match name in the paper (AppLoadBalancer).
Should also keep an eye out for the following:

- god config file
- nginx config file
- make sure nothing in load balancer is using absolute paths

Starting an AppController is done via ssh and forks into the background. If
it is killed, it prints that line to the screen. Need to find a way to
suppress this if possible. See below:

Nirvana@bahamut:~$ ruby appscale-terminate-instances.rb -ips ips-apc.yaml 
Shutting down AppScale components at 192.168.1.2
Shutting down AppScale components at 192.168.1.3
Shutting down AppScale components at 192.168.1.4
Shutting down AppScale components at 192.168.1.5
Nirvana@bahamut:~$ bash: line 1: 21583 Killed                  ruby
/root/appscale/AppController/djinnServer.rb > /tmp/192.168.1.2.log 2>&1

Unable to reset the password using the SOAP server.

Delete individual app or individual users

For MySQL the PBServer is not located at the head node, so refactoring is
needed to allow the pbserver to be anywhere.

- Changed code to reflect this, but has revealed another problem:
Cassandra runs a PBServer on all nodes, but the system only works if we
tell it to use the PBServer at the master. Currently investigating.

Currently the database master is always on the head node and the database
slaves are always on slave nodes. In the future this may not necessarily be
so, so if a database master/slave is not also hosting appserver or
apploadbalancer then we need to only allow access to it via the private ips
and disallow public ip access.

Waiting on Euca1.5 to see how networking support will be changed and how we
will decide when to separate DBs from ALB and AS.

[deleted issue]

Need an extra flag "private" that toggles whether or not we display the
app's name and link on the status page, and need to work it into
run-instances and upload-app (defaults to false). Need to also add extra
meta-data into soap server to store it in the DB.

[deleted issue]