Coder Social home page Coder Social logo

python-etcd's Introduction

python-etcd documentation

A python client for Etcd https://github.com/coreos/etcd

Official documentation: http://python-etcd.readthedocs.org/

https://travis-ci.org/jplana/python-etcd.png?branch=master https://coveralls.io/repos/jplana/python-etcd/badge.svg?branch=master&service=github

Installation

Pre-requirements

This version of python-etcd will only work correctly with the etcd server version 2.0.x or later. If you are running an older version of etcd, please use python-etcd 0.3.3 or earlier.

This client is known to work with python 2.7 and with python 3.3 or above. It is not tested or expected to work in more outdated versions of python.

From source

$ python setup.py install

From Pypi

$ python -m pip install python-etcd

Usage

The basic methods of the client have changed compared to previous versions, to reflect the new API structure; however a compatibility layer has been maintained so that you don't necessarily need to rewrite all your existing code.

Create a client object

import etcd

client = etcd.Client() # this will create a client against etcd server running on localhost on port 4001
client = etcd.Client(port=4002)
client = etcd.Client(host='127.0.0.1', port=4003)
client = etcd.Client(host=(('127.0.0.1', 4001), ('127.0.0.1', 4002), ('127.0.0.1', 4003)))
client = etcd.Client(host='127.0.0.1', port=4003, allow_redirect=False) # wont let you run sensitive commands on non-leader machines, default is true
# If you have defined a SRV record for _etcd._tcp.example.com pointing to the clients
client = etcd.Client(srv_domain='example.com', protocol="https")
# create a client against https://api.example.com:443/etcd
client = etcd.Client(host='api.example.com', protocol='https', port=443, version_prefix='/etcd')

Write a key

client.write('/nodes/n1', 1)
# with ttl
client.write('/nodes/n2', 2, ttl=4)  # sets the ttl to 4 seconds
client.set('/nodes/n2', 1) # Equivalent, for compatibility reasons.

Read a key

client.read('/nodes/n2').value
client.read('/nodes', recursive = True) #get all the values of a directory, recursively.
client.get('/nodes/n2').value

# raises etcd.EtcdKeyNotFound when key not found
try:
    client.read('/invalid/path')
except etcd.EtcdKeyNotFound:
    # do something
    print "error"

Delete a key

client.delete('/nodes/n1')

Atomic Compare and Swap

client.write('/nodes/n2', 2, prevValue = 4) # will set /nodes/n2 's value to 2 only if its previous value was 4 and
client.write('/nodes/n2', 2, prevExist = False) # will set /nodes/n2 's value to 2 only if the key did not exist before
client.write('/nodes/n2', 2, prevIndex = 30) # will set /nodes/n2 's value to 2 only if the key was last modified at index 30
client.test_and_set('/nodes/n2', 2, 4) #equivalent to client.write('/nodes/n2', 2, prevValue = 4)

You can also atomically update a result:

result = client.read('/foo')
print(result.value) # bar
result.value += u'bar'
updated = client.update(result) # if any other client wrote '/foo' in the meantime this will fail
print(updated.value) # barbar

Watch a key

client.read('/nodes/n1', wait = True) # will wait till the key is changed, and return once its changed
client.read('/nodes/n1', wait = True, timeout=30) # will wait till the key is changed, and return once its changed, or exit with an exception after 30 seconds.
client.read('/nodes/n1', wait = True, waitIndex = 10) # get all changes on this key starting from index 10
client.watch('/nodes/n1') #equivalent to client.read('/nodes/n1', wait = True)
client.watch('/nodes/n1', index = 10)

Refreshing key TTL

(Since etcd 2.3.0) Keys in etcd can be refreshed without notifying current watchers.

This can be achieved by setting the refresh to true when updating a TTL.

You cannot update the value of a key when refreshing it.

client.write('/nodes/n1', 'value', ttl=30)  # sets the ttl to 30 seconds
client.refresh('/nodes/n1', ttl=600)  # refresh ttl to 600 seconds, without notifying current watchers

Locking module

# Initialize the lock object:
# NOTE: this does not acquire a lock yet
client = etcd.Client()
# Or you can custom lock prefix, default is '/_locks/' if you are using HEAD
client = etcd.Client(lock_prefix='/my_etcd_root/_locks')
lock = etcd.Lock(client, 'my_lock_name')

# Use the lock object:
lock.acquire(blocking=True, # will block until the lock is acquired
      lock_ttl=None) # lock will live until we release it
lock.is_acquired  # True
lock.acquire(lock_ttl=60) # renew a lock
lock.release() # release an existing lock
lock.is_acquired  # False

# The lock object may also be used as a context manager:
client = etcd.Client()
with etcd.Lock(client, 'customer1') as my_lock:
    do_stuff()
    my_lock.is_acquired  # True
    my_lock.acquire(lock_ttl=60)
my_lock.is_acquired  # False

Get machines in the cluster

client.machines

Get leader of the cluster

client.leader

Generate a sequential key in a directory

x = client.write("/dir/name", "value", append=True)
print("generated key: " + x.key)
print("stored value: " + x.value)

List contents of a directory

#stick a couple values in the directory
client.write("/dir/name", "value1", append=True)
client.write("/dir/name", "value2", append=True)

directory = client.get("/dir/name")

# loop through directory children
for result in directory.children:
  print(result.key + ": " + result.value)

# or just get the first child value
print(directory.children.next().value)

Development setup

To check your code,

$ tox

to test you should have etcd available in your system path:

$ command -v etcd

to generate documentation,

$ cd docs
$ make

Release HOWTO

To make a release

  1. Update release date/version in NEWS.txt and setup.py
  2. Run 'python setup.py sdist'
  3. Test the generated source distribution in dist/
  4. Upload to PyPI: 'python setup.py sdist register upload'

python-etcd's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-etcd's Issues

mkdir fails

Under set a key in the docs it's mentioned that you can perform a mkdir operation:

# mkdir
client.write('/nodes/queue', dir=True)

That fails for me with

TypeError: write() takes at least 3 non-keyword arguments (2 given)

Prepare for a new release

We need to do the following:

  • Support calls to /stats (not sure it's fundamental, though)
  • Review docs
  • Model integration tests on the structure of the documented etcd API as much as possible

Then I'd say we're ready for a new release, as a ton of bugs have been resolved.

How to config etcd cluster

I try to build up etcd cluster according to:
https://github.com/coreos/etcd/blob/master/Documentation/clustering.md
The static way.

I run following command on 2 machines:
/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver1 -listen-peer-urls http://10.120.73.86:7001 -listen-client-urls http://10.120.73.86:4001 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.120.73.86:7001 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserve2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver2 -listen-peer-urls http://10.112.206.10:2380 -listen-client-urls http://10.112.206.10:2379 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.112.206.10:2380 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserver2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

But when I run client.machines. It will output
machines [u'http://localhost:2379', u'http://localhost:2379', u'http://localhost:4001', u'http://localhost:4001']
Seems it still find cluster machines from localhost.

Introduce logging

At the moment we do not have any logging ability and an user has no real way to understand what is going on in the library.

We should introduce some non-intrusive logging (so that at INFO nothing gets logged if all requests succeed).

TestClientRequest incorrectly inheriting from TestClientApiInterface?

I was trying to write some test cases for a bug I'm fixing and noticed that the TestClientApiInterface test cases were being run twice, with my new tests failing on the second run. I finally tracked the problem down to TestClientRequest inheriting from TestClientApiInterface, meaning that the TestClientApiInterface tests were being rerun as part of TestClientRequest.

Is TestClientRequest meant to being inheriting from TestClientApiBase instead?

Add support for client-side timeouts

Any etcd client should support the ability to close the HTTP request after a timeout; this is particularly important for watch and lock primitives.

Example

c = etcd.Client()
c.read('/testkey', wait=True, timeout=5000)
# we should also support this:
c.set_timeout(5000)
c.read('/testkey', wait=True)
  • Timeout should be in milliseconds, not seconds (if urllib3 allows that...)
  • If no timeout or 0 is provided, then no timeout will be applied.

Set doesn't fresh cache

commit: b4ff9ce

Assuming the value of '/omsstaging/db/name' is oms_staging

pprint(client.read('/omsstaging/db/name').value)
client.write('/omsstaging/db/name', u'oms_staging1')
pprint(client.read('/omsstaging/db/name').value)

This will output

u'oms_staging'
u'oms_staging'

Desired output should be

u'oms_staging'
u'oms_staging1'

watch with timeout=None, will raise urllib3.exceptions.ProtocolError

fields={'wait':'true'}, timeout=None)

File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 68, in request
File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 81, in request_encode_url
File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 153, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 530, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/response.py", line 284, in from_httplib
File "build/bdist.linux-x86_64/egg/urllib3/response.py", line 104, in init
File "build/bdist.linux-x86_64/egg/urllib3/response.py", line 214, in read
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

run in python2.7:
Python 2.7.8

Can't get root keys

Per the etcd team, I should be able to get the keys in the root:

curl -s http://localhost:4001/v2/keys/

It's not allowed with python-etcd:

>>> c.get('/')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/etcd/client.py", line 285, in get
    return self._result_from_response(response)
  File "/Library/Python/2.7/site-packages/etcd/client.py", line 327, in _result_from_response
    raise etcd.EtcdException('Unable to decode server response')
etcd.EtcdException: Unable to decode server response

Dustin

Available through PIP?

Only way to install is from source then? Just confirming because I see an etcd module via "pip search etcd".

Non-recusive read on empty dir gives the dir itself in it's children

I tried to make a simple function to find empty etcd folders like this:

def find_empty(etcd_client, key):
    empty = True
    logging.debug("checking key %s", key)
    for e in etcd_client.read(key).children:
        empty = False
        if e.dir:
            find_empty(etcd_client, e.key)
    if empty:
        logging.debug("Found empty key %s", key)

But this crashes with a maximum recursion depth exception rather than printing the message when it encounters an empty dir.

Project active?

We use this library in our company and I noticed this project hasn't had any updates in some time.. can anyone shed some light on if it is still actively maintained and will receive updates to ensure it continues working with future versions of etcd?

If not, can anyone suggest another python library?

Lock module support

I'm starting to work on support for the built-in lock module in etcd 0.2.0. I wanted to get feedback on what the API should look like. Here's my proposal:

    # Acquire a lock (with no value) for "customer1"
    index = client.acquire_lock('/customer1', ttl=60)
    with client.lock('/customer1', ttl=60):
        # lock is released when the `with` block closes
        do_something()

    # Acquire a lock for "customer1" that is associated with the value "bar"
    index = client.acquire_lock('/customer1', ttl=60, value='bar')
    with client.lock('/customer1', ttl=60, value='bar'):
        # lock is released when the `with` block closes
        do_something()

    # Renew the TTL on the "customer1" lock for index 2
    client.renew_lock('/customer1', ttl=60, index=2)

    # Renew the TTL on the "customer1" lock for value "bar"
    client.renew_lock('/customer1', ttl=60, value='bar')

    # Retrieve the current value for the "customer1" lock.
    client.get_lock_value('/customer1')

    # Retrieve the current index for the "customer1" lock
    client.get_lock_index('/customer1')

    # Delete the "customer1" lock with the index 2
    client.unlock('/customer1', index=2)

    # Delete the "customer1" lock with the value "bar"
    client.unlock('/customer1', value='bar')

The comments there are directly from etcd's documentation on the lock module.

My biggest question: would it be preferable to return some sort of EtcdLock object that has some data about the lock?

Keep in mind that etcd only returns the index of the lock when one is acquired - this is odd to me given that setting a normal value returns a lot more info.

Use PUT for sets rather than POST

It looks like you're considering PUT and POST to be analogous, at least in this [recent] pull request:

#14

However, POSTs are now used to enqueue values using the v2 API (see "In-Order Keys" in the etcd documentation). You should probably switch to preferring PUT exclusively.

client.update does not fail if value changed.

This block of code will correctly fail if the /test node is modified by something else between reading and writing:

x = client.read('/test')
time.sleep(5)
client.write('/test', int(x.value) + 1, prevIndex = x.modifiedIndex)

But this code succeeds, and overwrites the value that was set elsewhere:

x = client.read('/test')
time.sleep(5)
x.value = int(x.value) + 1
client.update(x)

If request timeout, Raise urllib3.exceptions.MaxRetryError not urllib3.exceptions.TimeoutError

fields={'wait':'true'}, timeout=1,)

File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 68, in request
File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 81, in request_encode_url
File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 153, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 581, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 581, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 581, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 561, in urlopen
File "build/bdist.linux-x86_64/egg/urllib3/util/retry.py", line 271, in increment
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='~~~', port=4001): Max retries exceeded with url: /v2/keys/test?wait=true (Caused by ReadTimeoutError("HTTPConnectionPool(host='~~~', port=4001): Read timed out.",))

and retries in urllib3.connectionpool default is urllib3.util.retry.Retry(3)

so, if watching a key with timeout, will fail?

python setup.py test fails in python 2.7

$ python setup.py test
running test
Searching for pyOpenSSL==0.13.1
Reading https://pypi.python.org/simple/pyOpenSSL/
Best match: pyOpenSSL 0.13.1
Downloading https://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.13.1.tar.gz#md5=e27a3b76734c39ea03952ca94cc56715
Processing pyOpenSSL-0.13.1.tar.gz
Writing /var/folders/2j/32g5k66x3db08z030c5nj0sh0000gn/T/easy_install-wreh2Y/pyOpenSSL-0.13.1/setup.cfg
Running pyOpenSSL-0.13.1/setup.py -q bdist_egg --dist-dir /var/folders/2j/32g5k66x3db08z030c5nj0sh0000gn/T/easy_install-wreh2Y/pyOpenSSL-0.13.1/egg-dist-tmp-uy8_8H
warning: no previously-included files matching '*.pyc' found anywhere in distribution

Installed /Users/jplana/src/etcd-python/python-etcd-github-temp2/pyOpenSSL-0.13.1-py2.7-macosx-10.4-x86_64.egg
running egg_info
writing requirements to src/python_etcd.egg-info/requires.txt
writing src/python_etcd.egg-info/PKG-INFO
writing top-level names to src/python_etcd.egg-info/top_level.txt
writing dependency_links to src/python_etcd.egg-info/dependency_links.txt
reading manifest file 'src/python_etcd.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/python_etcd.egg-info/SOURCES.txt'
running build_ext
Traceback (most recent call last):
  File "setup.py", line 36, in <module>
    test_suite='tests.unit',
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/Users/jplana/.pyenv/versions/temp2/lib/python2.7/site-packages/setuptools/command/test.py", line 138, in run
    self.with_project_on_sys_path(self.run_tests)
  File "/Users/jplana/.pyenv/versions/temp2/lib/python2.7/site-packages/setuptools/command/test.py", line 118, in with_project_on_sys_path
    func()
  File "/Users/jplana/.pyenv/versions/temp2/lib/python2.7/site-packages/setuptools/command/test.py", line 164, in run_tests
    testLoader = cks
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/main.py", line 94, in __init__
    self.parseArgs(argv)
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/main.py", line 149, in parseArgs
    self.createTests()
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/main.py", line 158, in createTests
    self.module)
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/loader.py", line 128, in loadTestsFromNames
    suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Users/jplana/.pyenv/versions/2.7.5/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
    module = __import__('.'.join(parts_copy))
ImportError: No module named tests

Not that I use it a lot, (travis will execute bin/test created by buildout) but It seems a nice to have.

[question] travis build of python-etcd

Hi,
I've noticed that you run your integration tests in travis, I am trying to do the same (for etcd-clojure) with my project.

I wonder how do you start the etcd server over there.

Thanks for your help

etcd published prefixes

I've been using python-etcd for all the client communication where the etcd is hosted remotely.

The python client connects to the server correctly when tried on a mac, but throws error on ubuntu.

The python code that I ran on both systems is:

import etcd
client = etcd.Client("55.88.55.161", 4001)
client.read("/_etcd/machines")

The error that ubuntu(in a VM and docker) threw was this:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 158, in __init__
    (self.__prefix, self.__machines))
ValueError: Could not identify given prefix [http://55.88.55.161:4001] among published prefixes: [[u'http://127.0.0.1:4001', None]]

Is there any way that the client treats the OS'es differently? Or is this an etcd quirk?

cluster's localhost IP must be 'localhost'

If you want to connect a cluster etcd,
you need to write the localhost's IP like 'localhost' as follow:

client = etcd.Client(
host=(
('localhost', 4001), //localhost IP
('192.168.1.33', 4001), //other machine'IP of the cluster
allow_reconnect=True,
)

you can‘t write the localhost's IP like '127.0.0.1' as follow, or the python-etcd will raise a exception:

client = etcd.Client(
host=(
('127.0.0.1', 4001),
('192.168.1.33', 4001),
allow_reconnect=True,
)

the problem is in python-etcd/client.py line 586:
self._machine_cache = self.machines
because the self.machines function return 'localhost' instead of '127.0.0.1' if any machine of the cluster is on your host

Any idea to solve the problem? I can write some code to fix it if anyone have a good idea.
Or at least mention that in the doc

Allow the write() method to accept an EtcdResult as input.

When using the library now, if you just need to update the TTL of a key, you need to do as follows:

res = c.read('/somekey')

c.write('/somekey', res.key, ttl=res.ttl+100, dir=res.dir)

It would make more sense, IMO, to be allowed to do as follows:

res = c.read('/somekey')
res.ttl += 100
c.write(res)

I find libraries not able to eat their own dogfood a little appalling.

What is your opinion?

Client.write() returns a bogus result when updating a dir in etcd > 0.2

Etcd v 0.3 introduced the "prevNode" property that is handled strangely for dirs; if we update a dir we obtain a strange result returned:

c = etcd.Client()
r = c.write('/foo', None, dir=True, ttl=50)
print(r.dir) #True
r2 = c.write('/foo', None, dir=True, ttl=120, prevExist=True)
print(r2.dir) #False

We should somehow use what is in prevNode to understand if the node is a dir.

I strongly suspect this is not the only case in which the write() response does something funny.

pyOpenSSL required only for tests

Hi,

pyOpenSSL library is required only for unit tests, but listed as a requirement for installation. This prevents installing python-etcd on RHEL5 which have outdated OpenSSL libraries.

Perhaps pyOpenSSL could be moved to test dependencies.

Support for creating directories

etcd API provides support for creating directories. Support for that seems missing in the current implementation of this library.

Or am I missing something? I haven't looked at the code in depth.

python etcd.EtcdException: Unable to decode server response

With etcd-v2.0.9-linux-amd64.
I run these 2 etcd process on two machine:
/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver1 -listen-peer-urls http://10.120.73.86:7001 -listen-client-urls http://10.120.73.86:4001 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.120.73.86:7001 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserve2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

/home/idcuser/HATest/etcd-v2.0.9-linux-amd64/etcd -name monitorserver2 -listen-peer-urls http://10.112.206.10:2380 -listen-client-urls http://10.112.206.10:2379 -initial-cluster-token monitorserver-cluster -initial-advertise-peer-urls http://10.112.206.10:2380 -initial-cluster monitorserver1=http://10.120.73.86:7001,monitorserver2=http://10.112.206.10:2380 -initial-cluster-state new &>> /home/idcuser/HATest/etcd.log &

Then in one machine 10.120.73.86:
I run following command through python:
etcd= etcd.Client(host='10.120.73.86', port=4001)
leader = etcd.read('/ms/leader/name' ).value
etcd.write( '/ms/leader/name', serverName, ttl=TTL )

Exception throws when I calling etcd.write( '/ms/leader/name', serverName, ttl=TTL )
Traceback (most recent call last):
File "/home/idcuser/HATest/HA_monitor2.python", line 118, in
main()
File "/home/idcuser/HATest/HA_monitor2.python", line 110, in main
run()
File "/home/idcuser/HATest/HA_monitor2.python", line 34, in run
leaderSelect()
File "/home/idcuser/HATest/HA_monitor2.python", line 50, in leaderSelect
beLeader()
File "/home/idcuser/HATest/HA_monitor2.python", line 66, in beLeader
etcd.write( '/ms/leader/name', serverName, ttl=TTL ) # value=xxxx
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 272, in write
response = self.api_execute(path, method, params=params)
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 587, in api_execute
return self.handle_server_response(response)
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 603, in handle_server_response
etcd.EtcdError.handle(**r)
File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/__init.py", line 184, in handle
raise exc(msg, payload)
etcd.EtcdException: Unable to decode server response

Read nodes as a tree.

EtcdResult has all of the children flattened, and sometimes I really want to traverse the result as a tree (the way that etcd provides them). Is this something you would consider?

Incorrect exceptions and timeouts with wait=True

I try to wait on a directory key with a timeout of, say 1s. If no updates to the key occur then the call times out after ~4s (not 1s as expected) and the exception is "EtcdException: No more machines in the cluster
" rather than a dedicated Timeout exception.

>>> import etcd
>>> c = etcd.Client(read_timeout=10)
>>> c.read("/foobar",waitIndex=88,wait=True,recursive=True,timeout=1)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 344, in read
    self.key_endpoint + key, self._MGET, params=params, timeout=timeout)
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 576, in api_execute
    self._base_uri = self._next_server()
  File "/usr/local/lib/python2.7/dist-packages/etcd/client.py", line 534, in _next_server
    raise etcd.EtcdException('No more machines in the cluster')
EtcdException: No more machines in the cluster
$ pip list | grep python-etcd
python-etcd (0.3.2)
$ pip list | grep urllib
urllib3 (1.10.2)

client hangs on write

Hi,

I have trouble when writing data to etcd although I can read data without any problem. Would anyone know where the problem could be?

I have following simple code:

#!/usr/bin/python
import etcd
                                                                                                                                                                   client = etcd.Client(host = '127.0.0.1', port = 5000)

print client.read('/coreos_apps/marathon/dev')
print client.write('/writetest', 'TESTSTRING')

Here is my output after I interrupt program with ctrl+C

-> % python scripts/etcd_test.py<class 'etcd.EtcdResult'>({'newKey': False, 'raft_index': 64071, '_children': [], 'createdIndex': 6684, 'modifiedIndex': 6684, 'value': u'{"ssh_port": 224, "count": 1, "version": "latest", "ssh_support": true}', 'etcd_index': 16626, 'expiration': None, 'key': u'/coreos_apps/marathon/dev', 'ttl': None, 'action': u'get', 'dir': False})




^CTraceback (most recent call last):
  File "scripts/etcd_test.py", line 9, in <module>
    print client.write('/writetest', 'TESTSTRING')
  File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 272, in write
    response = self.api_execute(path, method, params=params)
  File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 575, in api_execute
    redirect=self.allow_redirect)
  File "build/bdist.linux-x86_64/egg/urllib3/request.py", line 135, in request_encode_body
  File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 174, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/poolmanager.py", line 153, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 516, in urlopen
  File "build/bdist.linux-x86_64/egg/urllib3/connectionpool.py", line 308, in _make_request
  File "/usr/lib64/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/usr/lib64/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/usr/lib64/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.7/httplib.py", line 791, in send
    self.connect()
  File "build/bdist.linux-x86_64/egg/urllib3/connection.py", line 154, in connect
  File "build/bdist.linux-x86_64/egg/urllib3/connection.py", line 133, in _new_conn
  File "build/bdist.linux-x86_64/egg/urllib3/util/connection.py", line 78, in create_connection
  File "/usr/lib64/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
KeyboardInterrupt

Every time it seems it hangs here:

  File "/usr/lib/python2.7/site-packages/python_etcd-0.3.2-py2.7.egg/etcd/client.py", line 272, in write
    response = self.api_execute(path, method, params=params)

Could it be due to connection problem? Weird is that Read works without any problem.

I have no problems writing to database using curl.

My Setup:
python --version
Python 2.7.5

etcd version 0.4.6

Would you need any info for debugging ?

Include the etcd-index as returned in http headers in etcd-result

Our systems usually work by reading all the data for it's relevant subtree at startup and then wait for changes using read with wait and waitIndex. If this particular subtree has changed little then the modifiedIndexes of the elements in the subtree might be so small that passing them as waitIndex will not work. However we know the latest index at the time of the read is included in the http headers of the read. If we use that index as waitIndex to the read we know we will not miss changes and we are fairly certain that etcd will still have the relevant log. However, to do this we need to get this info out of the library

Streaming watch?

I have a proof-of-concept version of eternal_watch() that uses the streaming API, which I could clean up as a pull request. The streaming API is much more efficient, especially over TLS. Unfortunately, I think that API is 2.0.x+ only.

Are you interested in such a contribution? Do I need to make sure it supports older versions than 2.0.x?

client.watch doesn't work as expected.

I am trying the following as described here :

r = client.watch('/test_folder')

and then

client.set('/test_folder/test', 'abc')

Althought the value is set correctly the watch command never returned.

I tried to use

client.read('/test_folder', wait=True)

but I got the same results. :-(

Any suggestions?
Thank you.

Installation of python-etcd fails

Getting below error on ubuntu 14.04

No package 'libffi' found
Package libffi was not found in the pkg-config search path.
Perhaps you should add the directory containing libffi.pc' to the PKG_CONFIG_PATH environment variable No package 'libffi' found Package libffi was not found in the pkg-config search path. Perhaps you should add the directory containinglibffi.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libffi' found
compiling '_configtest.c':
__thread int some_threadlocal_variable_42;

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wa -Wstrict-prototypes -fPIC -c _configtest.c -o _configtest.o
success!
removing: _configtest.c _configtest.o
c/_cffi_backend.c:13:17: fatal error: ffi.h: No such file or directory
#include <ffi.h>
^
compilation terminated.
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Remove PyOpenSSL requirement

PyOpenSSL is only used to create snakeoil certificates during testing. Remove the certificate generation so this dependency, hard to install in the Windows platform, is not needed.

Installation failure on CoreOS

Trying to install python-etcd on Ubuntu within a Docker container on CoreOS.

Unfortunately, I am running into pyOpenSSL issues.

Prior to installing python-etcd. I installed OpenSSL (not sure if necessary) but failure below occurs with or without this step. Any advice is appreciated...

root@2a86bce32082:/etc/ibm/etech/coreos/python/python-etcd# openssl version
OpenSSL 1.0.1e 11 Feb 2013

root@2a86bce32082:/etc/ibm/etech/coreos/python# git clone https://github.com/jplana/python-etcd.git
Cloning into 'python-etcd'...
remote: Reusing existing pack: 529, done.
remote: Total 529 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (529/529), 96.02 KiB, done.
Resolving deltas: 100% (256/256), done.
root@2a86bce32082:/etc/ibm/etech/coreos/python# cd python-etcd
root@2a86bce32082:/etc/ibm/etech/coreos/python/python-etcd# python setup.py install
running install
running bdist_egg
running egg_info
creating src/python_etcd.egg-info
writing requirements to src/python_etcd.egg-info/requires.txt
writing src/python_etcd.egg-info/PKG-INFO
writing top-level names to src/python_etcd.egg-info/top_level.txt
writing dependency_links to src/python_etcd.egg-info/dependency_links.txt
writing manifest file 'src/python_etcd.egg-info/SOURCES.txt'
reading manifest file 'src/python_etcd.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'src/python_etcd.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/etcd
copying src/etcd/client.py -> build/lib.linux-x86_64-2.7/etcd
copying src/etcd/init.py -> build/lib.linux-x86_64-2.7/etcd
copying src/etcd/lock.py -> build/lib.linux-x86_64-2.7/etcd
creating build/lib.linux-x86_64-2.7/etcd/tests
copying src/etcd/tests/init.py -> build/lib.linux-x86_64-2.7/etcd/tests
creating build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/init.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/test_old_request.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/test_client.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
copying src/etcd/tests/unit/test_request.py -> build/lib.linux-x86_64-2.7/etcd/tests/unit
creating build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/test_simple.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/init.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/test_lock.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
copying src/etcd/tests/integration/helpers.py -> build/lib.linux-x86_64-2.7/etcd/tests/integration
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/etcd
copying build/lib.linux-x86_64-2.7/etcd/client.py -> build/bdist.linux-x86_64/egg/etcd
copying build/lib.linux-x86_64-2.7/etcd/init.py -> build/bdist.linux-x86_64/egg/etcd
copying build/lib.linux-x86_64-2.7/etcd/lock.py -> build/bdist.linux-x86_64/egg/etcd
creating build/bdist.linux-x86_64/egg/etcd/tests
copying build/lib.linux-x86_64-2.7/etcd/tests/init.py -> build/bdist.linux-x86_64/egg/etcd/tests
creating build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/init.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/test_old_request.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/test_client.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
copying build/lib.linux-x86_64-2.7/etcd/tests/unit/test_request.py -> build/bdist.linux-x86_64/egg/etcd/tests/unit
creating build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/test_simple.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/init.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/test_lock.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
copying build/lib.linux-x86_64-2.7/etcd/tests/integration/helpers.py -> build/bdist.linux-x86_64/egg/etcd/tests/integration
byte-compiling build/bdist.linux-x86_64/egg/etcd/client.py to client.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/lock.py to lock.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/test_old_request.py to test_old_request.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/test_client.py to test_client.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/unit/test_request.py to test_request.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/test_simple.py to test_simple.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/init.py to init.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/test_lock.py to test_lock.pyc
byte-compiling build/bdist.linux-x86_64/egg/etcd/tests/integration/helpers.py to helpers.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/python_etcd.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/python_etcd-0.3.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing python_etcd-0.3.0-py2.7.egg
creating /usr/local/lib/python2.7/dist-packages/python_etcd-0.3.0-py2.7.egg
Extracting python_etcd-0.3.0-py2.7.egg to /usr/local/lib/python2.7/dist-packages
Adding python-etcd 0.3.0 to easy-install.pth file

Installed /usr/local/lib/python2.7/dist-packages/python_etcd-0.3.0-py2.7.egg
Processing dependencies for python-etcd==0.3.0
Searching for pyOpenSSL==0.13.1
Reading https://pypi.python.org/simple/pyOpenSSL/
Best match: pyOpenSSL 0.13.1
Downloading https://pypi.python.org/packages/source/p/pyOpenSSL/pyOpenSSL-0.13.1.tar.gz#md5=e27a3b76734c39ea03952ca94cc56715
Processing pyOpenSSL-0.13.1.tar.gz
Writing /tmp/easy_install-4A9HDV/pyOpenSSL-0.13.1/setup.cfg
Running pyOpenSSL-0.13.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-4A9HDV/pyOpenSSL-0.13.1/egg-dist-tmp-BXlayj
warning: no previously-included files matching '*.pyc' found anywhere in distribution
OpenSSL/crypto/crypto.c:14:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: Setup script exited with error: command 'gcc' failed with exit status 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.