aagam41 / db_file_storage Goto Github PK
View Code? Open in Web Editor NEWCustom FILE_STORAGE for django. Saves your model's filefields in your database instead of your file system.
License: MIT License
Custom FILE_STORAGE for django. Saves your model's filefields in your database instead of your file system.
License: MIT License
Is there a way to get it work with Form wizards ?, I just added in my Form wizard class as:
file_storage = DatabaseFileStorage()
It raised this error:
list index out of range
The exception came from the method exists at line 104 in storage.py
I thought it was because I have to add the prefix like in upload_to: app_name/class/bytes/filename/mimetype, but I didn't know how to get it work.
If this issue doesn't go here, please drop it.
Thanks in advance.
With Django 1.9, getting error "db_file_storage/views.py", line 2, in from django.core.servers.basehttp import FileWrapper ImportError: cannot import name FileWrapper".
Should line 2 be replaced with "from wsgiref.util import FileWrapper"?
Library doesn't seem to work with django's ContentFile class, as described by @pwhipp on issue #2.
I think on line 60, mimetype = content.file.content_type
should be mimetype = content.content_type
.
I like to have Django admin support for all models in DEBUG
mode, as it helps understanding of the data model, making onboarding new devs easier.
The initial mixin could put the entire model into "read only" mode within Django admin - that is still useful.
Full edit support is easy for a text based file store.
The main field where editing is useful would be the mimetype. Sometimes that needs to be made more precise, e.g. using newly defined mimetypes like application/vnd.open-api+yaml
for old files which were a normal yaml mimetype.
Widgets for uploading raw binary data is a bit tricky, but is possible. e.g. https://stackoverflow.com/questions/58529099/adding-file-upload-widget-for-binaryfield-to-django-admin
However the admin for the FileModel could simply disable "add" until there is some way to upload the binary file.
https://github.com/victor-o-silva/db_file_storage/blob/master/db_file_storage/storage.py#L109
IMHO, the metadata (model class, field names etc) should not be saved in the filename
column.
Also, exists()
method does not work properly.
https://github.com/victor-o-silva/db_file_storage/blob/master/db_file_storage/storage.py#L136
The file is being looked up by it's correct name, but it's saved with the meta data.
With the changes added in 0.4.4, we were getting the following AttributeError
when using DBClearableFileInput
:
File "db_file_storage/form_widgets.py", line 35, in get_context
context['widget']['display'] = get_link_display(value.url)
AttributeError: 'NoneType' object has no attribute 'url'
The error occurs when trying to use this widget on a form with no values. We have fixed this on our end by creating a subclass of DBClearableFileInput
that does a basic check for value
when it is None
, but we wanted to point it out here as well.
Let us know if you need additional details. We appreciate the work you're doing here. Thanks!
After uploading file with russian letters in it's name (like 'персональный-рецепт-как-встретить-2016-год.jpg') using django admin:
Django 1.8.7, python 2.7.6 or 2.7.11, django-db-file-storage==0.3.3
After quite a bit of mucking around, I was able to get this working without putting DEFAULT_FILE_STORAGE = 'db_file_storage.storage.DatabaseFileStorage'
in my settings as is described in the README.
I wrote up my use case in a bit more detail at kimetrica/django-binary-database-files#31 which I was also experimenting with.
I found how to get it working with the use of storage=DatabaseStorage(..)
in their tests, and notice there isn't any similar tests or docs here which explain how to do that. See also kimetrica/django-binary-database-files#32 about that. I had seen storage
was available in the core Django field docs , but I tried quite a few incorrect incantations before realising how simple it was.
The models setup I have used is
from db_file_storage.storage import DatabaseFileStorage
class Filestore(models.Model):
data = models.BinaryField()
filename = models.CharField(max_length=255)
mimetype = models.CharField(max_length=50)
class Image(models.Model):
name = models.CharField(max_length=100)
image = models.ImageField(
storage=DatabaseFileStorage(),
upload_to='app.Filestore/data/filename/mimetype',
blank=True, null=True)
I think this should be explained in the docs, and have tests to make sure it works correctly without relying on DEFAULT_FILE_STORAGE
.
Templates are not bundled with current release.
Hi,
I#m trying to save a file with the help of the FileField Field. I'm not quiet sure, how to pass my file to the save function.
To test things I have a simple Model which uses the FileField:
class UserFiles(models.Model): file = models.FileField(upload_to='TestProject.UserFiles_FileModel/bytes/filename/mimetype') title = models.CharField(max_length=64)
and the corresponding FileModel:
class UserFiles_FileModel(models.Model): bytes = models.TextField() filename = models.CharField(max_length=255) mimetype = models.CharField(max_length=50)
I'm opening a testfile with
file = open("somefile.txt","r")
I'm then creating a new Table entry via:
entry = UserFiles(title="Sometitle")
I'm then trying to save the file with:
entry.file.save("somefile.txt",file)
Which gives back the Error: "TypeError: 'str' does not support the buffer interface"
Here is the stacktrace:
`---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
----> 1 entry.file.save("somefile.txt",file)
/usr/local/lib/python3.4/dist-packages/django/db/models/fields/files.py in save(self, name, content, save)
91
92 if func_supports_parameter(self.storage.save, 'max_length'):
---> 93 self.name = self.storage.save(name, content, max_length=self.field.max_length)
94 else:
95 warnings.warn(
/usr/local/lib/python3.4/dist-packages/django/core/files/storage.py in save(self, name, content, max_length)
61 name = self.get_available_name(name)
62
---> 63 name = self._save(name, content)
64
65 # Store filenames with forward slashes, even on Windows
/usr/local/lib/python3.4/dist-packages/db_file_storage/storage.py in _save(self, name, content)
116 new_filename = self._get_unique_filename(model_cls,
117 filename_field, name)
--> 118 encoded_bytes = self._get_encoded_bytes_from_file(content)
119 mimetype = getattr(content.file, 'content_type', 'text/plain')
120
/usr/local/lib/python3.4/dist-packages/db_file_storage/storage.py in _get_encoded_bytes_from_file(self, _file)
35 _file.seek(0)
36 file_content = _file.read()
---> 37 return base64.b64encode(file_content)
38
39 def _get_file_from_encoded_bytes(self, encoded_bytes):
/usr/lib/python3.4/base64.py in b64encode(s, altchars)
60 """
61 # Strip off the trailing newline
---> 62 encoded = binascii.b2a_base64(s)[:-1]
63 if altchars is not None:
64 assert len(altchars) == 2, repr(altchars)
`
add db_file_storage.model_utils.delete_file_if_needed o the docs
This module works great on my Linux dev box, but not on my Windows dev box. I am able to upload a file successfully (through the Admin app), but a GET request (or clicking the file download link in the Admin app) returns a 400 error with the message "Invalid request". I think the problem lies with the different path separators in the file name (back slash in Windows, forward slash in Linux), because when I change the direction of the path separators in the file name to back slashes in my GET requests on my Windows dev box, the file downloads perfectly:
this fails:
curl -X GET http://localhost:8000/files/download/?name=myapp.DatabaseFile/bytes/filename/mimetype/test.png
this works:
curl -X GET http://localhost:8000/files/download/?name=myapp.DatabaseFile\bytes\filename\mimetype\test.png
Also drop support for unsupported Django versions.
Exists any feature to enable cache header in response?
You accept pull request for this?
Hello,
I am facing an issue, I keep having the error TemplateDoesNotExist - db_file_storage/widgets/admin_clearable_file_input.html when I package my app.
I use other third party apps, no issue, all templates, static are collected and there, but for some reason not for db_file_storage. The app is properly referenced in INSTALLED_APPS
`Django tried loading these templates, in this order:
Using engine django:
django.template.loaders.filesystem.Loader: /Users/toto/.pex/install/Django-2.0.2-py3.6.egg.0a9fb7bc8d9dbc7082915e607b8e4ee0e96686d2/Django-2.0.2-py3.6.egg/django/forms/templates/db_file_storage/widgets/admin_clearable_file_input.html (Source does not exist)`
Django-2.0.2,
Python Version: 3.6.0
django-db-file-storage==0.5.1
any idea where it can come from ? thx
In March I was in a hurry and I locally fixed a path issue where Django always worked with '/' but db_file_storage worked with os.sep (I fixed it locally because it was above the @sp-niemand fork that fixed Django 1.9, so contributing the fix was not easy anyway.)
This is my fix:
https://github.com/SonOfLilit/db_file_storage/commits/master
I recently noticed that the issue was fixed upstream (as well as the Django 1.9 support that was blocking me from using the official release):
However, when I tested the new version, I found that the official fix converted Django's forward slashes to backslashes on Windows, instead of using forward slashes internally on Windows too like Django does, and like a db-based storage plugin should, so I'm stuck on my unofficial version.
The abstract issue is that there is no relation between a storage plugin and the operating system, so why should the storage plugin introduce a portability issue between operating systems, or depend on the operating system for it's API/ABI in any way?
An example concrete issue is that I can't clone the production DB and work on it from a Windows machine. Few people ever host production Django systems on a Windows server [citation needed], so dev environments for systems that are hosted on Unix in production is the major use case for using Django on Windows, and therefore this matters.
I propose moving to my fix and creating a migration that converts all existing paths in the DB to forward slashes (perhaps only do it if there are slashes in the config part of the stored paths... Windows does not allow backslashes in a filename, does Unix?).
I can write the code if you like the idea and are willing to do Code Review.
Aur
Hi how I can upload image / file though api ?
To be called after model_instance.delete()
Add it to the docs.
IMHO, it should be stored in the settings.
Something like:
{
'default': {} # config for the default storage,
'storage_1': {} # config for another storage
}
It looks like this project has had a few pull requests opened over the last couple of years, but they haven't been looked at. If maintenance is a burden, have you considered transferring this project over to Jazzband?
when using delete_file_if_needed to remove file, I got error as follows:
AttributeError: 'MyModel' object has no attribute 'id'
because I customized my model's primary_key name.
Is there planned support for Django 1.11 or should I try working on it myself?
Currently running the tests with Django 1.11.7 results with 3 failures. test_admin_form_widget_shows_proper_filename
, test_binary_file
, and test_form_widget_shows_proper_filename
.
Description
I'm sure this is an issue on my end but I can't figure out why Django is throwing an ImportError for db_file_storage.urls within my application's main urlpatterns (MyApp/urls.py). I've followed the configuration setup from here https://django-db-file-storage.readthedocs.io/en/master/ and I have the rest of the setup complete, I just can't seem to get past this Import issue.
When testing using the Django shell from CLI I am able to import 'db_file_storage.urls' fine and inspect the object. It's only when accessing the application from the browser does the Import fail to resolve and throws an error. I also tested importing db_file_storage at the top of my urls.py which was successful. Any attempts to include urls from db_file_storage resulted in the same Import error.
Environment
Django 2.0.1
Python 3.4.9 (Virtual Environment)
Apache 2.4.6 (CentOS)
Browser Error
ImportError at /
No module named 'db_file_storage.urls'
Request Method: | GET
Request URL: https://myapp.internalsite.com/
Django Version: 2.0.1
Exception Type: ImportError
Exception Value: No module named 'db_file_storage.urls'
Exception Location: <frozen importlib._bootstrap> in _find_and_load_unlocked, line 2224
Python Executable /usr/bin/python3
Python Version: 3.4.9
Python Path: ['/opt/MyProject',
'/opt/virtual-env/lib64/python34.zip',
'/opt/virtual-env/lib64/python3.4',
'/opt/virtual-env/lib64/python3.4/plat-linux',
'/opt/virtual-env/lib64/python3.4/lib-dynload',
'/usr/lib64/python3.4',
'/usr/lib/python3.4',
'/opt/virtual-env/lib/python3.4/site-packages']
urls.py (version 1) - throws ImportError: No module named 'db_file_storage.urls'
from django.contrib import admin
from django.urls import path, include
from django.conf.urls.static import static
from django.conf import settings
urlpatterns = [
path('', include('reports.urls')),
path('admin/', admin.site.urls),
path('findings/', include('findings.urls')),
path('reports/',include('reports.urls')),
path('accounts/', include('django.contrib.auth.urls')),
path('files/', include('db_file_storage.urls')),
] + static(settings.STATIC_URL,document_root=settings.STATIC_ROOT)
if settings.DEBUG:
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
urls.py (version 2) - Commented out the offending line and included an import of db_file_storage. Throws no errors, but doesn't include the necessary urls for db_file_storage
from django.contrib import admin
from django.urls import path, include
from django.conf.urls.static import static
from django.conf import settings
import db_file_storage
urlpatterns = [
path('', include('reports.urls')),
path('admin/', admin.site.urls),
path('findings/', include('findings.urls')),
path('reports/',include('reports.urls')),
path('accounts/', include('django.contrib.auth.urls')),
# path('files/', include('db_file_storage.urls')),
] + static(settings.STATIC_URL,document_root=settings.STATIC_ROOT)
if settings.DEBUG:
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
settings.py
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django_seo_js',
'findings',
'reports',
'sslserver',
'widget_tweaks',
'guardian',
'db_file_storage',
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
MIDDLEWARE_CLASSES = (
'django_seo_js.middleware.EscapedFragmentMiddleware', # If you're using #!
'django_seo_js.middleware.UserAgentMiddleware', # If you want to detect by user agent
)
ROOT_URLCONF = 'myproject.urls'
AUTHENTICATION_BACKENDS = (
'django.contrib.auth.backends.ModelBackend', # this is default
'guardian.backends.ObjectPermissionBackend',
)
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [os.path.join(BASE_DIR, 'templates')],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
'django.template.context_processors.request',
'django.template.context_processors.static',
],
},
},
]
WSGI_APPLICATION = 'myproject.wsgi.application'
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': '****',
'USER': '****',
'PASSWORD': '******',
'HOST': '******',
'PORT': 5432
}
}
# Password validation
# https://docs.djangoproject.com/en/2.0/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/2.0/howto/static-files/
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
DEFAULT_FILE_STORAGE = 'db_file_storage.storage.DatabaseFileStorage'
PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(PROJECT_ROOT, 'static')
httpd.conf - relevant configuration
IncludeOptional conf.d/*.conf
Alias /media/ /opt/MyProject/media/
Alias /static/ /opt/MyProject/myproject/static/
<Directory /opt/MyProject/myproject/static>
Require all granted
</Directory>
<Directory /opt/MyProject/media>
Require all granted
</Directory>
WSGIScriptAlias / /opt/MyProject/myproject/wsgi.py
LoadModule wsgi_module /opt/virtual-env/lib/python3.4/site-packages/mod_wsgi/server/mod_wsgi-py34.cpython-34m.so
WSGIDaemonProcess MyProject python-home=/opt/virtual-env/ python-path=/opt/MyProject:/opt/virtual-env/lib/python3.4/site-packages socket-user=nobody
WSGIProcessGroup MyProject
WSGISocketPrefix /var/run/wsgi
<Directory /opt/MyProject>
<Files wsgi.py>
Require all granted
</Files>
</Directory>
Since there is no other way to contact @Aagam41. I'm opening this issue to ask when can we expect a new release supporting the newer version of Django?
Not able to save files in windows environment.
ValueError('need more than 1 value to unpack',)
db_file_storage/storage.py in _get_storage_attributes, line 70
/Users/acer997/django-venv/src/django-db-file-storage/db_file_storage/urls.py:11: RemovedInDjango110Warning: Support for string view arguments to url() is deprecated and will be removed in Django 1.10 (got views.get_file). Pass the callable instead.
name='db_file_storage.download_file'
/Users/acer997/django-venv/src/django-db-file-storage/db_file_storage/urls.py:17: RemovedInDjango110Warning: Support for string view arguments to url() is deprecated and will be removed in Django 1.10 (got views.get_file). Pass the callable instead.
name='db_file_storage.get_file'
/Users/acer997/django-venv/src/django-db-file-storage/db_file_storage/urls.py:17: RemovedInDjango110Warning: django.conf.urls.patterns() is deprecated and will be removed in Django 1.10. Update your urlpatterns to be a list of django.conf.urls.url() instances instead.
name='db_file_storage.get_file'
/Users/acer997/django-venv/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1393: RuntimeWarning: DateTimeField PropertyRecord.date_uploaded received a naive datetime (2015-07-01 00:00:00) while time zone support is active.
RuntimeWarning)
Currently the file contents are stored in a TextField
I would like to store it in a bytea when using postgres.
Django has a BinaryField type (https://docs.djangoproject.com/en/2.1/ref/models/fields/#binaryfield)
Which should translate to bytea and the correct blob type for each database.
I believe this kind of behaviour could be configurable
remove license file
set appropriate classifier in setup.py
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.