pydanny / dj-libcloud

Adds easy python 3 and 2.7 support to Django for management of static assets.

Hi there 👋

• 🔭 I’m currently working for @octoenergy on decarbonizing the planet
• 😄 Pronouns: he/him
• 🐍 I code mostly in Python, but also some NodeJS and Rust
• 🌱 I’m currently learning Rust
• 🌎 Contact me if you want to join me in decarbonizing the planet
• 🥋 I'm currently training in Brazilian Jiu-Jitsu for sport and Tai Chi to help recover from a bike injury
• 📫 How to reach me:
  • mastodon
  • LinkedIn
  • twitter
• ⚡ Fun fact: I met my wife at PyCon US in 2010. She is awesome, her GH account is @audreyfeldroy

dj-libcloud's Issues

NotImplementedError: get_object_cdn_url() not implemented for this driver

Apparently, libcloud's S3StorageDriver class does not implement the get_object_cdn_url() method. I could not find it in https://github.com/apache/libcloud/blob/95b6a68adb5cbbe071d9da509484146cff6c5626/libcloud/storage/drivers/s3.py

You can see an example of the get_object_cdn_url() method in libcloud's CloudFilesStorageDriver (for Rackspace Cloudfiles) class at https://github.com/apache/libcloud/blob/d69adbd886965c7b176b9713cdad2f954d041c3c/libcloud/storage/drivers/cloudfiles.py#L298

The error:

Template error:
In template /vagrant/created/templates/base.html, error at line 26
   get_object_cdn_url not implemented for this driver

   26 :     <link href=" {% static 'css/project.css' %} " rel="stylesheet">

The traceback:

Traceback:
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  137.                 response = response.render()
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/response.py" in render
  105.             self.content = self.rendered_content
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/response.py" in rendered_content
  82.         content = template.render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/base.py" in render
  140.             return self._render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/base.py" in _render
  134.         return self.nodelist.render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/base.py" in render
  840.                 bit = self.render_node(node, context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/debug.py" in render_node
  78.             return node.render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  123.         return compiled_parent._render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/base.py" in _render
  134.         return self.nodelist.render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/base.py" in render
  840.                 bit = self.render_node(node, context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/debug.py" in render_node
  78.             return node.render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/loader_tags.py" in render
  62.             result = block.nodelist.render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/base.py" in render
  840.                 bit = self.render_node(node, context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/template/debug.py" in render_node
  78.             return node.render(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/templatetags/static.py" in render
  106.         url = self.url(context)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/django/contrib/staticfiles/templatetags/staticfiles.py" in url
  12.         return staticfiles_storage.url(path)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/djlibcloud/storage.py" in url
  169.         url = self.driver.get_object_cdn_url(obj)
File "/home/vagrant/.env/local/lib/python2.7/site-packages/libcloud/storage/base.py" in get_object_cdn_url
  296.             'get_object_cdn_url not implemented for this driver')

Exception Type: NotImplementedError at /
Exception Value: get_object_cdn_url not implemented for this driver

Is there a way we could work around this by constructing the URL some other way in dj-libcloud?

Speed it up. Seems to run really slow.

S3 Regions

I'm trying to use a bucket in EU-West-1.

I keep getting:
<LibcloudError in <class 'libcloud.storage.drivers.s3.S3StorageDriver'> 'This bucket is located in a different region. Please use the correct driver.'>

I can't find a way to change the driver. Settings only seem to allow a generic S3 driver, with no option for region. Am I missing something?

Add better error message if bucket not found

CachedFilesMixin class breaks the second time you run it.

This is the problem code:

    from django.contrib.staticfiles.storage import CachedFilesMixin

    from djlibcloud.storage import LibCloudStorage
    from pipeline.storage import PipelineMixin

   class PipelineCachedCloudStorage(PipelineMixin,
                                 CachedFilesMixin,
                                 LibCloudStorage):
    """ Borks on second run of collectstatic"""
    pass

Failed when I tried it without the PipelineMixin, but haven't tested that comprehensively.

In reference to issue #12

So I decided to try and use dj-libcloud for my default file storage since I saw that you implemented the .url feature in issue #12 , which does not work in django-storages.

The good news is that I can upload files to google cloud all day long with dj-libcloud, unfortunately when I try to use {{ foo.bar.url }} in a template, it returns None instead of the expected url.

Any thoughts or ideas?

Tests! OMG TESTS!!!

We need to:

• Determine if we will use a mock system or actually hit the APIs
• Make the test runner actually work. See #8

Remove the dependency on django-pipeline

Come up with more storage types so we aren't dependant on django-pipeline. Nothing wrong with pipeline, just want to provide more options.

Add wheels!

Backport to Python 2.7

Get travis.ci working

[django-configurations] Values not evaluated with Python 2.7

The values that are given to dj-libcloud from the settings files are configuration value objects, not strings. This causes all kinds of problems. Current mitigation is to replace SecretValues interacting with dj-libcloud with get_env_variable functions:

# Normally you should not import ANYTHING from Django directly
# into your settings, but ImproperlyConfigured is an exception.
from django.core.exceptions import ImproperlyConfigured

def get_env_variable(var_name):
    """Get the environment variable or return exception."""
    try:
        return os.environ[var_name]
    except KeyError:
        error_msg = "Set the %s environment variable" % var_name
        raise ImproperlyConfigured(error_msg)

class Production(Common):
    AWS_ACCESS_KEY_ID = get_env_variable("DJANGO_AWS_ACCESS_KEY_ID")
    AWS_SECRET_ACCESS_KEY = get_env_variable("DJANGO_AWS_SECRET_ACCESS_KEY")
    AWS_STORAGE_BUCKET_NAME = get_env_variable("DJANGO_AWS_STORAGE_BUCKET_NAME")

Ping @jezdez.

Convert over to py.test

Cause I like it.

file.url is slow - libcloud performs a HEAD on each file

Related to issue #5, when using Amazon S3 as the storage backend it performs a HEAD request:

def url(self, name):
    provider_type = self.provider['type'].lower()
    obj = self._get_object(name)

I'd suggest adding another (optional) setting for each libcloud provider, something like:

LIBCLOUD_PROVIDERS = {
    'default': {
        ...
        'url_prefix': 'http://example.org/',
    },
}

If it exists, then just combine url_prefix + name to give the URL to a file. If the setting isn't added then use the old method of doing a request to figure out the URL.