noxiouz / docker-registry-driver-elliptics Goto Github PK

It works perfectly with 22.6.3.37, but not with 2.26.3.38. At the first glance Elliptics *.38 uses new version of Blackhole, so there're some changes in configuration. May be something else.

Elliptics has an ability to read bytes with offset and size. Also it has prepare-write-commit way to write data by chunks. So it's better to use this features to reduce unnecessary memory usage.

On first glance implementation of streaming reading won't lead to any problems. but when it comes to write operation, some underwater rocks could be found.

Hi!

I'm trying to build working docker-registry docker image with your elliptics storage driver. Here is my progress on Ubuntu 14.04 LTS:

cd /tmp
git clone https://github.com/dotcloud/docker-registry.git
cd docker-registry
sudo apt-get install elliptics-client
sudo pip install .
sudo pip install docker-registry-driver-elliptics

All looks fine, but when I try to start registry with SETTINGS_FLAVOR=elliptics, I got following:

2014-05-21 01:07:25 [22765] [INFO] Starting gunicorn 18.0
2014-05-21 01:07:25 [22765] [INFO] Listening at: http://0.0.0.0:5000 (22765)
2014-05-21 01:07:25 [22765] [INFO] Using worker: gevent
2014-05-21 01:07:25 [22770] [INFO] Booting worker with pid: 22770
2014-05-21 01:07:25,648 DEBUG: Will return docker-registry.drivers.elliptics.Storage
2014-05-21 01:07:25 [22770] [ERROR] Exception in worker process:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 495, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 106, in init_process
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 114, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 62, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 49, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 354, in import_app
    __import__(module)
  File "/usr/local/lib/python2.7/dist-packages/docker_registry/wsgi.py", line 7, in <module>
    from .run import app
  File "/usr/local/lib/python2.7/dist-packages/docker_registry/run.py", line 16, in <module>
    from .tags import *  # noqa
  File "/usr/local/lib/python2.7/dist-packages/docker_registry/tags.py", line 21, in <module>
    store = storage.load()
  File "/usr/local/lib/python2.7/dist-packages/docker_registry/storage/__init__.py", line 36, in load
    _storage[kind] = engine.fetch(kind)(None, config=cfg)
  File "/usr/local/lib/python2.7/dist-packages/docker_registry/drivers/elliptics.py", line 45, in __init__
    cfg.config.wait_timeout = config.get("elliptics_wait_timeout", 60)
ArgumentError: Python argument types in
    None.None(Config, str)
did not match C++ signature:
    None(dnet_config {lvalue}, unsigned int)

I've try in python console:

/tmp/docker-registry$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import elliptics
>>> cfg = elliptics.Config()
>>> cfg.config.wait_timeout = 100
>>> cfg.config.wait_timeout
100
>>> 

So it looks like error relates to config.get("elliptics_wait_timeout", 60) part of line. I think that something changed in docker-registry Config interface, but I can't dig so deep in code...

I found following line in docker-registry CHANGELOG.md: largely enhanced configuration mechanism (setup-configs.sh is no more).

I use docker-registry commit #0f1523bfd704affa61f1a63650c6ced45bc04e56

Any idias?

Thanks

Title says all.

Having the ability to run against a mocked elliptics is a plus.

Given elliptics (python package as well) is installed from deb (system-wide) we can't test other python versions easily.
We either need a pip installable package for elliptics (pyclient), or a (preferably simple) way to build it from source in each venv.

For some reason, python from inside the venv in tox can't find the system-wide installed elliptics package (though tox is instructed to look system-wide).

Thus, no test run on travis right now.

Hey there :-)

Have a look here:
docker-archive/docker-registry#444

How does this concern you?

• myvar = cfg.get(key, DEFAULT) no longer exist - just replace these calls with myvar = cfg.key or DEFAULT for the same effect
• values (be they default in config.yaml or sourced from ENV) are now typed properly (eg: they are YAML evaluated), so, you likely can drop the explicit casts you had to introduce earlier

This will likely get released with registry 0.8 by the end of the month.

These changes are breaking drivers, so I decided to bump the core version (2.0.0) to ease the transition (though nothing in core really changed): just up your dependency to the core and you should be done.

I encourage you to mention the change as well in your README, so that people looking for a 0.7 compatible release can pip install an old version of your driver (or you decide maintaining 0.7 compatibility is not worth it and drop it).

Hope that helps!

Cheers.

Hello, I was trying to deploy Docker Registry server with Elliptics backend inside Docker container, but the configuration example that you provide is no longer suitable. This is the one I ended up with - need to change the elliptics_verbosity value:

elliptics:
    <<: *common
    storage: elliptics
    elliptics_nodes: ["localhost:1025:2"]
    elliptics_wait_timeout: 60
    elliptics_check_timeout: 60
    elliptics_io_thread_num: 2
    elliptics_net_thread_num: 2
    elliptics_nonblocking_io_thread_num: 2
    elliptics_groups: [2]
    elliptics_verbosity: "debug"
    elliptics_logfile: "/var/log/docker-registry-elliptics.log"

Tested with the following packages:

# dpkg -l | grep elliptics
ii  elliptics                              2.26.3.33                        amd64        Distributed hash table storage
ii  elliptics-client                       2.26.3.33                        amd64        Distributed hash table storage (client library)
# pip freeze | grep docker
docker-registry==0.9.1
docker-registry-core==2.0.3
docker-registry-driver-elliptics==0.1.1

There appears to be an error building the elliptics driver:

Installed /usr/local/lib/python2.7/dist-packages/docker_registry_driver_elliptics-0.0.4-py2.7.egg
Processing dependencies for docker-registry-driver-elliptics==0.0.4
Searching for elliptics>=2
Reading https://pypi.python.org/simple/elliptics/
Couldn't find index page for 'elliptics' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
No local packages or download links found for elliptics>=2
error: Could not find suitable distribution for Requirement.parse('elliptics>=2')