metagriffin / asset

A python package resource and symbol loading helper library.

Generalized Package Asset Loader

Loads resources and symbols from a python package, whether installed as a directory, an egg, or in source form. Also provides some other package-related helper methods, including asset.version(), asset.caller(), and asset.chunks().

TL;DR

Install:

$ pip install asset

Load symbols (e.g. functions, classes, or variables) from a package by name:

import asset

# load the 'mypackage.foo.myfunc' function and call it with some parameter
retval = asset.symbol('mypackage.foo.myfunc')(param='value')

Load data files from a package:

# load the file 'mypackage/templates/data.txt' into string
data = asset.load('mypackage:templates/data.txt').read()

# or as a file-like stream
stream = asset.load('mypackage:templates/data.txt').stream()
data   = stream.read()

Multiple files can be operated on at once by using globre style wildcards:

# concatenate all 'css' files into one string:
css = asset.load('mypackage:static/style/**.css').read()

# load all '.txt' files, XML-escaping the data and wrapping
# each file in an <node name="...">...</node> element.
import xml.etree.ElementTree as ET
data = ET.Element('nodes')
for item in asset.load('asset:**.txt'):
  cur = ET.SubElement(data, 'node', name=item.name)
  cur.text = item.read()
data = ET.tostring(data)

Query the installed version of a package:

asset.version('asset')
# ==> '0.0.5'

asset.version('python')
# ==> '2.7'

asset.version('no-such-package')
# ==> None

Find out what package is calling the current function:

# assuming the call stack is:
#   in package "zig" a function "x", which calls
#   in package "bar" a function "y", which calls
#   in package "foo" a function "callfoo" defined as:

def callfoo():

  asset.caller()
  # ==> 'bar'

  asset.caller(ignore='bar')
  # ==> 'zig'

  asset.caller(ignore=['bar', 'zig'])
  # ==> None

Call all the plugins for a given group:

for plugin in asset.plugins('mypackage.plugins'):
  plugin.handle()

Filter an object through all the plugins for a given group (if there are no plugins, this will simply return thing):

result = asset.plugins('mypackage.plugins').filter(thing)

Load all registered plugins, select the ones named foo and invoke them (this will fail if there is no foo plugin):

result = asset.plugins('mypackage.plugins').select('foo').handle(thing)

Chunk a file (or any file-like object) into 1 KiB chunks:

with open('/var/binary/data', 'rb') as fp:
  for chunk in asset.chunks(fp, 1024):
    # ... do something with `chunk` ...

Chunk an Asset stream (here using the .chunks alias method):

for chunk in asset.load('mypackage:data/**.bin').chunks():
  # ... using the default chunk size (usually 8 KiB) ...

Testing

In order to run the unit tests correctly, the pxml package needs to be installed as a zipped package (i.e. an "egg") and the globre package needs to be installed unzipped. To accomplish that, do:

$ easy_install --zip-ok pxml
$ easy_install --always-unzip globre

The reason is that the unit tests confirm that asset can load assets from both zipped and unzipped packages, and can also identify in which mode it is operating.

Details

TODO: add detailed docs...

• Asset.filename:

  If the asset represents a file on the filesystem, is the absolute path to the specified file. Otherwise is None.

• AssetGroupStream.readline():

  Returns the next line from the aggregate asset group stream, as if the assets had been concatenate into a single asset.

  IMPORTANT: if an asset ends with content that is not terminated by an EOL token, it is returned as-is, i.e. it does NOT append the first line from the next asset.

Note: because asset.load() does lazy-loading, it only throws a NoSuchAsset exception when you actually attempt to use the AssetGroup! If you need an immediate error, use the peek() method. Note that it returns itself, so you can do something like:

import asset

def my_function_that_returns_an_iterable():

  return asset.load(my_spec).peek()

  # this returns exactly the same thing as the following:
  #
  #   return asset.load(my_spec)
  #
  # but throws an exception early if there are no matching assets.