tmetsch / occi-os Goto Github PK

Hi,
I'm trying to create an instance of the network type but when i send a POST request the server answers me "Currently not supported."
I'm on openstack grizzly and I'm working a program, written in python, that will test the compliance of openstack grizzly whith the OCCI standard.
I need to create a network instance, how can I do?
Thanks a lot.

OCCI reports that the floating IPs has been detached - but it is actually still attached to the VM

Hi there,

the workflow of allocating floating IPs is sligthly different in Nova and OCCI. Whereas in OCCI, it is done in one step (action=alloc_float_ip), you need to allocate an IP to the project first and only then associate it with a machine.

When terminating a machine through OCCI (or Nova if you like), the IP remains allocated to the project. A new OCCI alloc_float_ip will allocate a new address to the project, thus consuming all available IPs over time.

The desired behaviour would be to also deallocate IPs from the project, thus making the actions symmetric.

Cheers,
Björn

See: https://bugs.launchpad.net/openstack-ci/+bug/1172884

Stackforge: http://ci.openstack.org/stackforge.html

1. create a VM
2. creat sec group
3. Associate sec group with VM
   4...won't work... :-(

Due to interoperability issues with other OCCI implementations, we have enable automatic assignment of floating IPs to our instances. This leads to the following problem:

1. create VM
2. delete VM fails because of the auto-assigned floating IP

2013-10-10 15:20:33.248 INFO nova.occiapi.wsgi.server [req-7a2c63f6-1089-4a78-885c-6f2f04ff46fd ridGermany/OU=Forschungszentrum Juelich GmbH/CN=Bjoern Hagemeier df37f5b1ebc94604964c2854b9c0551f] 134.94.168.82,127.0.0.1 "DELETE /compute/7b1ea357-aa52-4774-a554-ecd690883daf HTTP/1.0" status: 400 len: 220 time: 0.2978840

This is what the client receives in the response body:

<- "DELETE /compute/ecd49c64-dbf1-40bd-b351-45e0c88a48b1 HTTP/1.1\r\nAccept: text/plain,text/occi\r\nUser-Agent: rOCCI HTTPClient 4.1.0\r\nX-Auth-Token: *****\r\nConnection: close\r\nHost: egi-cloud.zam.kfa-juelich.de:8787\r\n\r\n"
-> "HTTP/1.1 400 Bad Request\r\n"
-> "Date: Thu, 10 Oct 2013 13:26:39 GMT\r\n"
-> "Server: pyssf OCCI/1.1\r\n"
-> "Content-Length: 46\r\n"
-> "Content-Type: text/plain\r\n"
-> "Vary: Accept-Encoding\r\n"
-> "Connection: close\r\n"
-> "\r\n"
reading 46 bytes...
-> ""
-> "Cannot disassociate auto assigined floating ip"

1. create VM
2. delete auto-assigned floating IP link

2013-10-10 15:22:31.138 INFO nova.occiapi.wsgi.server [req-57f1904a-e9f7-4e43-b1b4-c52a7f187740 ridGermany/OU=Forschungszentrum Juelich GmbH/CN=Bjoern Hagemeier df37f5b1ebc94604964c2854b9c0551f] 134.94.168.82,127.0.0.1 "DELETE /network/interface/bbc1b1fe-aca5-4f30-8cbb-12f36af9a163 HTTP/1.0" status: 400 len: 220 time: 0.3253710

<- "DELETE /network/interface/545f8ecb-bac0-4b11-899c-b4e78eb93ce0 HTTP/1.1\r\nAccept: text/plain,text/occi\r\nUser-Agent: rOCCI HTTPClient 4.1.0\r\nX-Auth-Token: *****\r\nConnection: close\r\nHost: egi-cloud.zam.kfa-juelich.de:8787\r\n\r\n"
-> "HTTP/1.1 400 Bad Request\r\n"
-> "Date: Thu, 10 Oct 2013 13:28:45 GMT\r\n"
-> "Server: pyssf OCCI/1.1\r\n"
-> "Content-Length: 46\r\n"
-> "Content-Type: text/plain\r\n"
-> "Vary: Accept-Encoding\r\n"
-> "Connection: close\r\n"
-> "\r\n"
reading 46 bytes...
-> ""
-> "Cannot disassociate auto assigined floating ip"

3. delete VM SUCCESSFUL
   2013-10-10 15:22:33.161 INFO nova.occiapi.wsgi.server [req-c659e1ec-ff34-40b3-9606-dcac15e9dc2c ridGermany/OU=Forschungszentrum Juelich GmbH/CN=Bjoern Hagemeier df37f5b1ebc94604964c2854b9c0551f] 134.94.168.82,127.0.0.1 "DELETE /compute/f0e37ded-e6a2-4361-bd73-c74637a66998 HTTP/1.0" status: 200 len: 146 time: 0.3598518

So, despite the deletion of the networklink being reported as failed, something changed, such that the VM can be deleted afterwards.

Hi there,

it is stated [1] that SSH keys and administrator passwords can be injected into an instance when it gets created. I may be blind, but I don't see how this can be done. I do have registered public SSH keys in my user/tenant, but don't know how I would inject them via the OCCI interface. Could you point me to some documentation, please.

My assumption would be that SSH keys would be added as a mixin, but I can't find any.

BTW, the documentation seems a bit dated. Am I looking at the wrong page?

Cheers,
Björn

[1] https://wiki.openstack.org/wiki/Occi#Create_a_VM_2

Currently the OCCI API maintains its own in-memory registry of resources. A new pyssf registry should be implemented to read from the openstack DB.

Might be an nova bug - needs further investigation.

@dizz can you check if it works for you?

Hi there,

when retrieving the list of available images, all registered images will be returned, even those that the current user should not see.

This may be a security issue, as it exposes information that should not be available (existence of images belonging to other tenants).

Secondly, when trying to instantiate such an image, the instantiation will fail, which is at least annoying for the user.

Björn

Console links are still present after deleting an VM.

QI will still show the sec group

Currently i setup a openstack environment with occi-os and i get an error after i installed and configured occi-os.

From nova-api.log:

2013-07-30 11:11:02.769 1102 CRITICAL nova [-] cannot import name flags
2013-07-30 11:11:02.769 1102 TRACE nova Traceback (most recent call last):
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/bin/nova-api", line 61, in
2013-07-30 11:11:02.769 1102 TRACE nova server = service.WSGIService(api, use_ssl=should_use_ssl)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/service.py", line 598, in init
2013-07-30 11:11:02.769 1102 TRACE nova self.app = self.loader.load_app(name)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/nova/wsgi.py", line 482, in load_app
2013-07-30 11:11:02.769 1102 TRACE nova return deploy.loadapp("config:%s" % self.config_path, name=name)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2013-07-30 11:11:02.769 1102 TRACE nova return loadobj(APP, uri, name=name, *_kw)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2013-07-30 11:11:02.769 1102 TRACE nova return context.create()
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 710, in create
2013-07-30 11:11:02.769 1102 TRACE nova return self.object_type.invoke(self)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2013-07-30 11:11:02.769 1102 TRACE nova *_context.local_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/util.py", line 56, in fix_call
2013-07-30 11:11:02.769 1102 TRACE nova val = callable(_args, *_kw)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 25, in urlmap_factory
2013-07-30 11:11:02.769 1102 TRACE nova app = loader.get_app(app_name, global_conf=global_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2013-07-30 11:11:02.769 1102 TRACE nova name=name, global_conf=global_conf).create()
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 362, in app_context
2013-07-30 11:11:02.769 1102 TRACE nova APP, name=name, global_conf=global_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 450, in get_context
2013-07-30 11:11:02.769 1102 TRACE nova global_additions=global_additions)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 559, in _pipeline_app_context
2013-07-30 11:11:02.769 1102 TRACE nova APP, pipeline[-1], global_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 454, in get_context
2013-07-30 11:11:02.769 1102 TRACE nova section)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 476, in _context_from_use
2013-07-30 11:11:02.769 1102 TRACE nova object_type, name=use, global_conf=global_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 406, in get_context
2013-07-30 11:11:02.769 1102 TRACE nova global_conf=global_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
2013-07-30 11:11:02.769 1102 TRACE nova global_conf=global_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 328, in _loadegg
2013-07-30 11:11:02.769 1102 TRACE nova return loader.get_context(object_type, name, global_conf)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 620, in get_context
2013-07-30 11:11:02.769 1102 TRACE nova object_type, name=name)
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 646, in find_egg_entry_point
2013-07-30 11:11:02.769 1102 TRACE nova possible.append((entry.load(), protocol, entry.name))
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2013, in load
2013-07-30 11:11:02.769 1102 TRACE nova entry = import(self.module_name, globals(),globals(), ['name'])
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/local/lib/python2.7/dist-packages/occi_os_grizzly-1.0-py2.7.egg/occi_os_api/init.py", line 40, in
2013-07-30 11:11:02.769 1102 TRACE nova from occi_os_api import wsgi
2013-07-30 11:11:02.769 1102 TRACE nova File "/usr/local/lib/python2.7/dist-packages/occi_os_grizzly-1.0-py2.7.egg/occi_os_api/wsgi.py", line 28, in
2013-07-30 11:11:02.769 1102 TRACE nova from nova import flags
2013-07-30 11:11:02.769 1102 TRACE nova ImportError: cannot import name flags
2013-07-30 11:11:02.769 1102 TRACE nova

@dizz: can you test?

Don't know if this is intended, but I'd think it would improve usability.

Naturally, the titles of existing storages created through other interfaces cannot be retrieved either.

Cheers,
Björn

Hi guys,

is there some contextualization handling designed into OCCI ? (I think to metadata service/cloudinit implementation in Openstack)

OCCI request on the details of a specific VM (example [1]) takes a very long time to complete (let say around 60s), while the same request through Nova API is executed instantly. Digging into the code, I figured that a request made in nova_glue/vm.py, function get_vnc, was timeouting when the VNC console configuration is not properly set on the Openstack nova installation:

console = COMPUTE_API.get_vnc_console(context, instance, 'novnc')

fails with message "Console info is not available yet!". Is that more a configuration issue of the openstack cluster or should the code handle that case ?

[1] curl -X GET $ACCESS_URL/compute/$INSTANCE_UUID

Although requested under /mgroup/ it will show up under /securtity/my_grp

Should be:

• openstackocci for latest and greatest
• openstackocci-essex for the essex release
• openstackocci-folsom for the folsom release
• openstackocci-grizzly for the grizzly release

Horzion interface shows error - but the OCCI interface will return an active. Minor fix: change when the states gets updated in the compute backend!

If you create a storage link with Horizon, and then inspect the storage links with

curl -v -X GET localhost:8787/storage/link/ -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID

and it won't be provisioned if you look for everything the occi offers.
On the other hand when you create the storage link through OCCI it will be shown in Horizon and it will be provisioned by OCCI.

Hi,

after fixing #32, we're not quite there yet. Even the patch I provided to do a more complete mapping of invalid characters onto valid ones, does not seem to be complete. rOCCI client returns the following problem:

$ occi --endpoint https://egi-cloud.zam.kfa-juelich.de:8787/ --auth x509
--resource storage --action list --user-cred /tmp/x509up_u500 --proxy-ca
~/.globus/usercert.pem

line 0:-1 mismatched input "" expecting CATEGORY_KEY
line 1:152 mismatched character "u"; expecting "k"
line 1:162 mismatched character "r"; expecting "t"
line 1:313 mismatched character "u"; expecting "k"
line 1:329 mismatched character "r"; expecting "t"
line 1:139 mismatched character "u"; expecting "k"
line 1:287 mismatched character "u"; expecting "k"

Storage locations:

@arax is aware of this and investigating what exactly the cause for this is, i.e. which are the conflicting characters.

We certainly need the reverse mapping of the category terms, before that it's useless to dig any further.

Hi there,

when querying resources on a system, the image categories can contain spaces in them. For example:

Category: OpenSuSE 12.1 Minimal; scheme="http://schemas.openstack.org/template/os#"; class="mixin"; title="This is an OS OpenSuSE 12.1 Minimal VM image"; rel="http://schemas.ogf.org/occi/infrastructure#os_tpl"; location="https://myhost.mydomain:1234/OpenSuSE 12.1 Minimal/"

Should encoding be used to avoid these spaces? They don't seem to comply with the specification.

Best regards,
Björn

look into how exception will be thrown - how message propegated and when messages logged!

Minor issue in the representation of the default tnetwork instance.

namely:

• pep8
• pylint
• coverage
• pyflakes
• nosetests

The action to allocate a floating ip is somehow not provisioned anymore and even if you associate a floating ip through Openstack directly, you can't see it listed in details of a vm.

so people can get up and running quickly

Hi there,

I know it is a bad idea to delete images that are still in use. We've done it anyway and see the following problem now. Maybe OCCI can fail a little more gracefully here, e.g. like Horizon simply says that the image was not found, but otherwise continue.

Cheers,
Björn

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 382, in handle_one_response
result = self.application(self.environ, start_response)
File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in call
return app(environ, start_response)
File "/usr/lib/python2.7/dist-packages/keystone/middleware/auth_token.py", line 278, in call
return self.app(env, start_response)
File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 159, in call
return resp(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/occi_os_folsom-1.0-py2.7.egg/occi_os_api/wsgi.py", line 159, in call
registry=self.registry)
File "/usr/local/lib/python2.7/dist-packages/occi/wsgi.py", line 228, in _call_occi
status, headers, body = handler.handle(mtd, key)
File "/usr/local/lib/python2.7/dist-packages/occi/handlers.py", line 68, in handle
return getattr(self, str.lower(method))(key)
File "/usr/local/lib/python2.7/dist-packages/occi/handlers.py", line 320, in get
self.extras)
File "/usr/local/lib/python2.7/dist-packages/occi/workflow.py", line 276, in get_entities_under_path
for res in registry.get_resources(extras):
File "/usr/local/lib/python2.7/dist-packages/occi_os_folsom-1.0-py2.7.egg/occi_os_api/registry.py", line 259, in get_resources
ent_list = self._construct_occi_compute(item['uuid'], extras)
File "/usr/local/lib/python2.7/dist-packages/occi_os_folsom-1.0-py2.7.egg/occi_os_api/registry.py", line 309, in _construct_occi_compute
image_id = storage.get_image(os_id, context)['id']
File "/usr/local/lib/python2.7/dist-packages/occi_os_folsom-1.0-py2.7.egg/occi_os_api/nova_glue/storage.py", line 113, in get_image
return IMAGE_API.show(context, uid)
File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 195, in show
_reraise_translated_image_exception(image_id)
File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 193, in show
image = self._client.call(context, 1, 'get', image_id)
File "/usr/lib/python2.7/dist-packages/nova/image/glance.py", line 138, in call
return getattr(client.images, method)(_args, *_kwargs)
File "/usr/lib/python2.7/dist-packages/glanceclient/v1/images.py", line 96, in get
resp, body = self.api.raw_request('HEAD', '/v1/images/%s' % image_id)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 191, in raw_request
return self._http_request(url, method, **kwargs)
File "/usr/lib/python2.7/dist-packages/glanceclient/common/http.py", line 158, in _http_request
raise exc.from_response(resp)
ImageNotFound: Image 9134eeb8-8631-41e1-b50c-1139c17fc30d could not be found.

I am using the latest version (master branch) of occi but it seems that when you delete a volume it hast issues with string indices that are supposed to be integers. Does anybody else have this issue?

curl -v -X DELETE http://localhost:8787/storage/162515a7-d02b-42b7-9fd7-4a5728d14a6d -H 'X-Auth-Token: '$KID -H 'Content-Type: text/occi'

< HTTP/1.1 400 Bad Request
< Content-Length: 31
< Content-Type: text/plain
< Content-Length: 31
< Server: pyssf OCCI/1.1
< Date: Wed, 31 Jul 2013 08:06:26 GMT
<

• Connection #0 to host localhost left intact
• Closing connection #0
  string indices must be integers

Some assignments are made to the compute backend - should be checked! https://github.com/tmetsch/occi-os/blob/master/occi_os_api/wsgi.py#L128

When i try to create a new Instance through curl it throws this issue

'module' object has no attribute 'get_instance_type_by_flavor_id'

CURL:

curl -v -X POST 141.52.208.91:8787/compute/ -H 'Category: compute; scheme="http://schemas.ogf.org/occi/infrastructure#"; class="kind"' -H 'Content-Type: text/occi' -H 'X-Auth-Token: '$KID -H 'Category: cirros-0-3-1-x86_64-uec; scheme="http://schemas.openstack.org/template/os#"; class="mixin"; title="This is an OS cirros-0.3.1-x86_64-uec VM image"; rel="http://schemas.ogf.org/occi/infrastructure#os_tpl"; location="http://141.52.208.91:8787/cirros-0-3-1-x86_64-uec/"' -H 'Category: m1-xlarge; scheme="http://schemas.openstack.org/template/resource#"; class="mixin"; title="This is an openstack m1.xlarge flavor."; rel="http://schemas.ogf.org/occi/infrastructure#resource_tpl"; location="http://141.52.208.91:8787/m1-xlarge/"'

Creating a new VM works through Horizon, so i presume, that it might be an issue with occi.

Hi,

I've been pointed to this issue about the occi.core.id. @arax is of the opinion that it should not, but I cannot see anything in the core OCCI spec that would restrict the id. After all, it is a URI w/o semantics. Let's clarify this.

Björn

1. Create a snapshot
2. See what will be provisioned to occi
3. Delete the snapshot
4. Deleted snapshot still provisioned

TEMPORARY FIX: restart nova-api

Possible fix: automatic restart of nova-api (but I'm sure that there are more elegant ways to deal with this.)

Hi,

after successful creation of a storage link, I am unable to delete it through OCCI. This is what I get in return:

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/eventlet/wsgi.py", line 383, in handle_one_response
result = self.application(self.environ, start_response)
File "/usr/lib/python2.7/dist-packages/paste/urlmap.py", line 203, in call
return app(environ, start_response)
File "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py", line 450, in call
File "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in call
return resp(environ, start_response)
File "/usr/local/lib/python2.7/dist-packages/openstackocci_grizzly-1.0-py2.7.egg/occi_os_api/wsgi.py", line 159, in call
registry=self.registry)
File "/usr/local/lib/python2.7/dist-packages/occi/wsgi.py", line 232, in _call_occi
status, headers, body = handler.handle(mtd, key)
File "/usr/local/lib/python2.7/dist-packages/occi/handlers.py", line 69, in handle
return getattr(self, str.lower(method))(key)
File "/usr/local/lib/python2.7/dist-packages/occi/handlers.py", line 296, in delete
workflow.delete_entity(entity, self.registry, self.extras)
File "/usr/local/lib/python2.7/dist-packages/occi/workflow.py", line 95, in delete_entity
entity.source.links.remove(entity)
ValueError: list.remove(x): x not in list

Cheers,
Björn

Update to reflect changes in implementation - e.g. floating IPs.

Hi there,

when deleting a flavor via nova flavor-delete, I still see them listed via OCCI. Newly created flavors show all right.

The old flavors go away when restarting the API server.

Cheers,
Björn

If you delete a snapshot or a storage link of OpenStack through Horizon or Terminal, you can still see them in the list of provisions.

curl -v -H 'X-Auth-Token: '$KID -X GET localhost:8787/-/

the snapshots are saved in the mySQL database but set as inactive/deleted. Where is the code snippet that retrieves the images and snapshots?

We should probably build a filter for active images into the code.

Dear All,

I use a laptop with Ubuntu 12.04, have followed the OCCI for OpenStack guide https://wiki.openstack.org/wiki/Occi. Concretely I ran the following commads:
-> pip install pyssf
-> pip install openstackocci
-> git clone git://github.com/openstack-dev/devstack.git
-> ./stack.sh

After installation, when trying to run curl -v -H 'X-Auth-Token: '$KID -X GET localhost:8787/-/ , an error message appears: " Trying 127.0.0.1... Connection refused" . I have checked the port 8787 using netstat. The port 8787 isn't active. It seems that the OCCI has not been installed correctly.

The config-, log- and output files are availabe here: https://www.dropbox.com/s/r2a0o2px3ln5moq/installtion_info.zip

What did I do wrong?

Thanks in advance,

Yongzheng Liang

SSH console will point to: org.openstack.compute.console.ssh ssh://:22 :-/

tests needs to be updated to cover changes from latest commits. @bhagemeier, alvarolopez can you maybe help?

is there anyway to name an instance when creating it, that i have not yet discovered?

Why? Obvious :-p but will also test OCCI network semantics in the infrastructure spec.

Enhancement, as dicussed at OGF 40.