clusterhq / pyzfs Goto Github PK

Python bindings to the stable ZFS ABI

License: Apache License 2.0

Python 100.00%

pyzfs's Introduction

This package provides a wrapper for libzfs_core C library.

libzfs_core is intended to be a stable interface for programmatic
administration of ZFS.
This wrapper provides one-to-one wrappers for libzfs_core API functions,
but the signatures and types are more natural to Python.
nvlists are wrapped as dictionaries or lists depending on their usage.
Some parameters have default values depending on typical use for
increased convenience.
Enumerations and bit flags become strings and lists of strings in Python.
Errors are reported as exceptions rather than integer errno-style
error codes.  The wrapper takes care to provide one-to-many mapping
of the error codes to the exceptions by interpreting a context
in which the error code is produced.

Unit tests and automated test for the libzfs_core API are provided
with this package.
Please note that the API tests perform lots of ZFS dataset level
operations and ZFS tries hard to ensure that any modifications
do reach stable storage. That means that the operations are done
synchronously and that, for example, disk caches are flushed.
Thus, the tests can be very slow on real hardware.
It is recommended to place the default temporary directory or
a temporary directory specified by, for instance, TMP environment
variable on a memory backed filesystem.

Package documentation: http://pyzfs.readthedocs.org
Package development: https://github.com/ClusterHQ/pyzfs

pyzfs's People

Contributors

Stargazers

Watchers

Forkers

carlcarl baits loli10k mikoim sriramnrn gotcramer avg-i rezroo leonwl kaveman- ihavec

pyzfs's Issues

Project NO LONGER MAINTAINED

Unfortunately, this project is no longer maintained and no one has write access to this repo any more. The code has moved to: https://github.com/zfsonlinux/zfs/tree/master/contrib/pyzfs

dataset names not accepted

Hi,
i was playing around with pyzfs and noticed this:

Python 3.3.5 (default, Mar 9 2016, 01:03:10)
[GCC 4.9.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import libzfs_core
>>> libzfs_core.lzc_list_snaps(bytes('pool/dataset', 'utf-8'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.3/site-packages/libzfs_core/_libzfs_core.py", line 722, in _f
return func(*args, **kwargs)
File "/usr/lib64/python3.3/site-packages/libzfs_core/_libzfs_core.py", line 1077, in lzc_list_snaps
for entry in _list(name, recurse=1, types=['snapshot']):
File "/usr/lib64/python3.3/site-packages/libzfs_core/_libzfs_core.py", line 948, in _list
(fd, other_fd) = lzc_list(name, options)
File "/usr/lib64/python3.3/site-packages/libzfs_core/_libzfs_core.py", line 722, in _f
return func(*args, **kwargs)
File "/usr/lib64/python3.3/site-packages/libzfs_core/_libzfs_core.py", line 906, in lzc_list
opts_nv = nvlist_in(options)
File "/usr/lib64/python3.3/site-packages/libzfs_core/_nvlist.py", line 58, in nvlist_in
_dict_to_nvlist(props, nvlist)
File "/usr/lib64/python3.3/site-packages/libzfs_core/_nvlist.py", line 233, in _dict_to_nvlist
raise TypeError('Unsupported key type ' + type(k).__name__)
TypeError: Unsupported key type str

# pip show pyzfs
---
Metadata-Version: 1.1
Name: pyzfs
Version: 0.2.2
Summary: Wrapper for libzfs_core
Home-page: http://pyzfs.readthedocs.org
Author: ClusterHQ
Author-email: [email protected]
License: Apache License, Version 2.0
Location: /usr/lib64/python3.3/site-packages
Requires: cffi

Is my call of lzc_list_snaps incorrect or is this a bug?

monkey-patching functools.wraps in sphinx conf.py is an ugly hack

Replacing functools.wraps decorator is an ugly hack.
Consider finding a different solution or manually documenting the affected code.

better way to convert between Python dicts and nvlists (in and out)

Currently a programmer has to use nvlist_in and nvlist_out that introduce Python objects corresponding to C nvlist_t objects (or placeholders) to a local namespace.
It would be better to use a proxy object that would transparently convert the Python dict arguments as
necessary that invoke a proper C function (via cffi) passing appropriate nvlist_t * and nvlist_t ** arguments to it. cffi provides necessary introspection tools to find out which arguments need translation.
An example of this idea could be seen here: ac179a2

libzfs_core.lzc_get_props() returns wrong value for parent dataset if child datasets are present

If libzfs_core.lzc_get_props(b'parent-dataset') is executed, it returns properties for the first child dataset instead of the parents own properties. If more than one child is present it returns the properties of the first child listed by libzfs_core.lzc_list_children(b'parent-dataset').

My environment is running python2.7 with pyzfs-0.2.2 and zfs-0.6.5.4-r2 on Gentoo.

Best regards

pyzfs fails on OSX

I'm trying to use pyzfs as a universal frontend to zfs on OSX, Linux and FreeBSD. Not really sure if it can/should work in this manner, but at least on my macbook I get the errors below. The ZFS implementation is https://openzfsonosx.org/ .. Any help is appreciated

$ python3
Python 3.5.2 (default, Jul 28 2016, 21:28:00)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import libzfs_core as z
>>> z.lzc_list_snaps('zsource')
Traceback (most recent call last):
  File "/private/tmp/src/agent/lib/python3.5/site-packages/cffi/api.py", line 757, in _load_backend_lib
    raise OSError("library not found: %r" % (name,))
OSError: library not found: 'zfs_core'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/tmp/src/agent/lib/python3.5/site-packages/libzfs_core/_libzfs_core.py", line 720, in _f
    if not is_supported(_f):
  File "/private/tmp/src/agent/lib/python3.5/site-packages/libzfs_core/_libzfs_core.py", line 691, in is_supported
    return is_supported(check_func)
  File "/private/tmp/src/agent/lib/python3.5/site-packages/libzfs_core/_libzfs_core.py", line 692, in is_supported
    return getattr(_lib, fname, None) is not None
  File "/private/tmp/src/agent/lib/python3.5/site-packages/libzfs_core/_libzfs_core.py", line 1103, in __getattr__
    ret = self._lib.libzfs_core_init()
  File "/private/tmp/src/agent/lib/python3.5/site-packages/libzfs_core/bindings/__init__.py", line 28, in __getattr__
    self._lib = self._ffi.dlopen(self._libname)
  File "/private/tmp/src/agent/lib/python3.5/site-packages/cffi/api.py", line 139, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
  File "/private/tmp/src/agent/lib/python3.5/site-packages/cffi/api.py", line 769, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
  File "/private/tmp/src/agent/lib/python3.5/site-packages/cffi/api.py", line 764, in _load_backend_lib
    return backend.load_library(path, flags)
OSError: cannot load library /usr/local/lib/libzfs_core.dylib: dlopen(/usr/local/lib/libzfs_core.dylib, 2): Symbol not found: _zfs_flags
  Referenced from: /usr/local/lib/libzfs_core.dylib
  Expected in: flat namespace
 in /usr/local/lib/libzfs_core.dylib

dynamically map Python str flags to C enum flags

Rather than doing this:

for flag in flags:
    c_flag = {
        'embedded_data':    _lib.LZC_SEND_FLAG_EMBED_DATA,
        'large_blocks':     _lib.LZC_SEND_FLAG_LARGE_BLOCK,
    }.get(flag)
    ...

we can algorithmically map the original strings to string corresponding to enum names in the C code and then use getattr(_lib, ...) to obtain actual values.

there is no way to pass an empty array via Python dict to C nvlist

C code allows to place an empty array of a specified type into an nvlist_t object,
but on the Python side it is currently impossible to deduce the intended value type
from an empty array. Also, at present there is no way to provide the type explicitly.

use versioneer for better version data management

Use versioneer for better version data management.

nvlist: mixing Python boolean values and boolean_t in the same array is not supported

The following code would raise an exception:

props = {"key": [boolean_t(False), True]}
nv_in = nvlist_in(props)

NotImplementedError: lzc_get_props ??

I wanned to try pyzfs to check if I could replace my "zfs" command line by pyzfs call. I've install pyzfs using pip2 install pyzfs on a system with zfs on linux.

In a python prompt:

>>> libzfs_core.lzc_get_props('tank')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/libzfs_core/_libzfs_core.py", line 721, in _f
    raise NotImplementedError(func.__name__)
NotImplementedError: lzc_get_props

Could I get more explanation about this exception ? Is it some compatibility issue with zfs onlinux ? I also tried libzfs_core.lzc_create('tank/test2') and it's working. Not sure where to go from here.

split libzfs_core test-cases into multiple fine-grained classes

Each class should test a single libzfs_core interface method.