Coder Social home page Coder Social logo

django-field-history's Introduction

django-field-history

Documentation Status https://travis-ci.org/grantmcconnaughey/django-field-history.svg?branch=master https://coveralls.io/repos/github/grantmcconnaughey/django-field-history/badge.svg?branch=master

A Django app to track changes to a model field. For Python 2.7/3.4+ and Django 1.11/2.0+.

Other similar apps are django-reversion and django-simple-history, which track all model fields.

Project django-field-history django-reversion django-simple-history
Admin Integration N/A Yes Yes
All/Some fields Some Some All
Object History No Yes Yes
Model History N/A No Yes
Multi-object Revisions N/A Yes No
Extra Model Manager Yes No Yes
Model Registry No Yes No
Django View Helpers No Yes No
Manager Helper Methods N/A Yes Yes (as_of, most_recent)
MySQL Support Extra config Complete Complete

Documentation

The full documentation is at https://django-field-history.readthedocs.io.

Features

  • Keeps a history of all changes to a particular model's field.
  • Stores the field's name, value, date and time of change, and the user that changed it.
  • Works with all model field types (except ManyToManyField).

Quickstart

Install django-field-history:

pip install django-field-history

Be sure to put it in INSTALLED_APPS.

INSTALLED_APPS = [
    # other apps...
    'field_history',
]

Then add it to your models.

from field_history.tracker import FieldHistoryTracker

class PizzaOrder(models.Model):
    STATUS_CHOICES = (
        ('ORDERED', 'Ordered'),
        ('COOKING', 'Cooking'),
        ('COMPLETE', 'Complete'),
    )
    status = models.CharField(max_length=64, choices=STATUS_CHOICES)

    field_history = FieldHistoryTracker(['status'])

Now each time you change the order's status field information about that change will be stored in the database.

from field_history.models import FieldHistory

# No FieldHistory objects yet
assert FieldHistory.objects.count() == 0

# Creating an object will make one
pizza_order = PizzaOrder.objects.create(status='ORDERED')
assert FieldHistory.objects.count() == 1

# This object has some fields on it
history = FieldHistory.objects.get()
assert history.object == pizza_order
assert history.field_name == 'status'
assert history.field_value == 'ORDERED'
assert history.date_created is not None

# You can query FieldHistory using the get_{field_name}_history()
# method added to your model
histories = pizza_order.get_status_history()
assert list(FieldHistory.objects.all()) == list(histories)

# Or using the custom FieldHistory manager
histories2 = FieldHistory.objects.get_for_model_and_field(pizza_order, 'status')
assert list(histories) == list(histories2)

# Updating that particular field creates a new FieldHistory
pizza_order.status = 'COOKING'
pizza_order.save()
assert FieldHistory.objects.count() == 2

updated_history = histories.latest()
assert updated_history.object == pizza_order
assert updated_history.field_name == 'status'
assert updated_history.field_value == 'COOKING'
assert updated_history.date_created is not None

Management Commands

django-field-history comes with a few management commands.

createinitialfieldhistory

This command will inspect all of the models in your application and create FieldHistory objects for the models that have a FieldHistoryTracker. Run this the first time you install django-field-history.

python manage.py createinitialfieldhistory

renamefieldhistory

Use this command after changing a model field name of a field you track with FieldHistoryTracker:

python manage.py renamefieldhistory --model=app_label.model_name --from_field=old_field_name --to_field=new_field_name

For instance, if you have this model:

class Person(models.Model):
    username = models.CharField(max_length=255)

    field_history = FieldHistoryTracker(['username'])

And you change the username field name to handle:

class Person(models.Model):
    handle = models.CharField(max_length=255)

    field_history = FieldHistoryTracker(['handle'])

You will need to also update the field_name value in all FieldHistory objects that point to this model:

python manage.py renamefieldhistory --model=myapp.Person --from_field=username --to_field=handle

Storing Which User Changed the Field

There are two ways to store the user that changed your model field. The simplest way is to use the logged in user that made the request. To do this, add the FieldHistoryMiddleware class to your MIDDLEWARE setting.

MIDDLEWARE = [
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'field_history.middleware.FieldHistoryMiddleware',
]

Alternatively, you can add a _field_history_user property to the model that has fields you are tracking. This property should return the user you would like stored on FieldHistory when your field is updated.

class Pizza(models.Model):
    name = models.CharField(max_length=255)
    updated_by = models.ForeignKey('auth.User')

    field_history = FieldHistoryTracker(['name'])

    @property
    def _field_history_user(self):
        return self.updated_by

Working with MySQL

If you're using MySQL, the default configuration will throw an exception when you run migrations. (By default, FieldHistory.object_id is implemented as a TextField for flexibility, but indexed columns in MySQL InnoDB tables may be a maximum of 767 bytes.) To fix this, you can set FIELD_HISTORY_OBJECT_ID_TYPE in settings.py to override the default field type with one that meets MySQL's constraints. FIELD_HISTORY_OBJECT_ID_TYPE may be set to either:

  1. the Django model field class you wish to use, or
  2. a tuple (field_class, kwargs), where field_class is a Django model field class and kwargs is a dict of arguments to pass to the field class constructor.

To approximate the default behavior for Postgres when using MySQL, configure object_id to use a CharField by adding the following to settings.py:

from django.db import models
FIELD_HISTORY_OBJECT_ID_TYPE = (models.CharField, {'max_length': 100})

FIELD_HISTORY_OBJECT_ID_TYPE also allows you to use a field type that's more efficient for your use case, even if you're using Postgres (or a similarly unconstrained database). For example, if you always let Django auto-create an id field (implemented internally as an AutoField), setting FIELD_HISTORY_OBJECT_ID_TYPE to IntegerField will result in efficiency gains (both in time and space). This would look like:

from django.db import models
FIELD_HISTORY_OBJECT_ID_TYPE = models.IntegerField

Running Tests

Does the code actually work?

source <YOURVIRTUALENV>/bin/activate
(myenv) $ pip install -r requirements-test.txt
(myenv) $ python runtests.py

django-field-history's People

Contributors

adamchainz avatar blag avatar grantmcconnaughey avatar mariodev avatar matthewslaney avatar mscansian avatar ramusus avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

django-field-history's Issues

Temporarily disabling field tracking

Is there any way we can temporarily disable tracking either by disconnecting signals or using some magic settings?
Example reason for doing this would be to pull historical data (timestamp, value) from already existing source. I'm not sure this is possible at this point..

@grantmcconnaughey I'll be glad to make PR + docs, after we figure out the way to handle this.
Thanks!

JSONB

In postgresql there is specific and optimized field for JSON, jsonb which can permit the direct used of the json object (and i guess that other dbb has his own).
The created table stock in serialized_ data the object. I think it could really improve the lib efficiency if the field type could set himself regarding project configuration.

All fields

Please add keyword for track all fields, like a __all__

relation "field_history_fieldhistory" does not exist on createinitialfieldhistory

Hello!

I am using PostgreSQL and I'm getting the following error upon python manage.py createinitialfieldhistory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 28, in <module>
    execute_from_command_line(sys.argv)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/core/management/base.py", line 353, in execute
    output = self.handle(*args, **options)
  File "/home/vagrant/grt/lib/python3.6/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 37, in handle
    field_name=field).exists():
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/query.py", line 715, in exists
    return self.query.has_results(using=self.db)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/query.py", line 516, in has_results
    return compiler.has_results()
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1031, in has_results
    return bool(self.execute_sql(SINGLE))
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1061, in execute_sql
    cursor.execute(sql, params)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/vagrant/grt/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "field_history_fieldhistory" does not exist
LINE 1: SELECT (1) AS "a" FROM "field_history_fieldhistory" WHERE ("...

my model:

class Title(models.Model):
    id = models.PositiveIntegerField(primary_key=True)
    name = models.CharField(max_length=255) 
    priority = models.PositiveIntegerField(null=True, blank=True)
    field_history = FieldHistoryTracker(['priority'])

tried doing this as well: https://django-field-history.readthedocs.io/en/latest/readme.html#working-with-mysql (just in case) - no success.
What could be wrong?

edit:
you have to use python manage.py migrate your new field first

ManyToManyField inside tracked field breaks field_value function

Consider the following models:

class OKB_Article(models.Model):
    VISUALIZATION_CHOICES = (
        ('is_private', 'Privado'),
        ('is_public', 'Publico'),
    )
    id_record = models.BigIntegerField()
    title = models.CharField(max_length=200)
    visualization = models.CharField(
        max_length=50, choices=VISUALIZATION_CHOICES, default='is_private')
    article_grouping = models.ForeignKey(
            OKB_ArticleGrouping,
            blank=True,
            null=True)
    author = models.ForeignKey(User, related_name='user_author')
    motive_type = models.ForeignKey(OSD_Motive, blank=True, null=True)
    article_type = models.ForeignKey(OKB_ArticleType, blank=True, null=True)
    content = models.TextField()
    approved_by = models.ForeignKey(
            User, related_name='approved_by', blank=True, null=True)
    created_date = models.DateTimeField(auto_now_add=True)
    status_type = models.ForeignKey(OSD_Status)

    history = FieldHistoryTracker([
        'motive_type', 'article_grouping', 'article_type',
        'visualization', 'status_type'
        ])

    class Meta:
        db_table = 'okb_article'

    def __unicode__(self):
        return self.title
class OSD_Status(models.Model):
    name = models.CharField(max_length=50)
    color = models.CharField(max_length=6, choices=COLOR_CHOICES,
                             default="ffce93")
    description = models.CharField(max_length=200)
    behavior = models.ForeignKey(OSD_Behavior)
    motive = models.ManyToManyField(OSD_Motive, through='OSD_StatusMotive')
    status = models.BooleanField(default=True)

    class Meta:
        db_table = 'osd_status'

    def __unicode__(self):
        return "%s - %s" % (self.name, self.behavior)

As you can see the field status_type in OKB_Article has a ManyToManyField called motive.

If I'm tracking the status_type field, the field_value function only works for that field; I can't access the field_value of any other field, it throws a DeserializationError mentioning the status_type field even though I'm not accesing it. See the screenshot where I reproduce and mimick what the field_value function does inside an ipdb debugger:

image

However, this only happens when I'm tracking said field. If I remove it from the FieldHistoryTracker list, I can access all the other fields without a problem.

I'm using the latest version of Django 1.9.x along with the latest Python 2.7.x on Linux.

Python 2.7.6 Django 1.7 Can't run createinitialfieldhistory command

$ ./manage.py createinitialfieldhistory
Traceback (most recent call last):
File "src/manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 399, in execute_from_command_line
utility.execute()
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 272, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/core/management/init.py", line 75, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
import(name)
File "/home/vagrant/mdm/env/local/lib/python2.7/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 4, in
from django.apps import apps
ImportError: No module named apps

Have install/quickstart docs mention migration

Not a huge problem but it would be good to have the installation and quickstart sections of the docs mention that you need to run manage.py migrate after adding 'field_history' to INSTALLED_APPS. Would save django newbies and people who don't spend a ton of time with it a minute or two of head-scratching, and is more technically complete.

renamefieldhistory does not change serialized_data

I would like to rename the field "status" to "portal_status" in my app. Originally this is what it looks like:

image

I executed python manage.py renamefieldhistory --model=accounts.account --from_field=status --to_field=portal_status and saw that 619 rows were updated.

Upon closer inspection however, I see that field_name is changed but not the field inside serialized_data.

image

This causes my application to crash.

Perhaps we should also parse serialized_data to and modify the keys accordingly?

automatics history table creation

missing this feature on the project, which could be very usefull. Or at last the documentation must explain how the history table looks like and his behavior. Anyway good job and good luck/

Problematic is_new_object based on primary key with default uuid

Hi.

There is a problem with Models which have primary key with default=uuid

id = models.UUIDField(default=uuid.uuid4, editable=False, primary_key=True)

The initial/first field history value is not saved to database.
It is caused by is_new_object = instance.pk is None in tracker.py

This approach has a problem:
In the save method, self.pk will never be None when default=uuid.uuid is set.

A large number of django users (and a lot of non-official django tutorials) believe that checking self.pk in the save method is a safe way to detect and decide whether an instance of a model is new or not.

No its not. The safe way to detect and decide whether an instance of a model is new or not is to use self._state object
https://docs.djangoproject.com/en/3.2/ref/models/instances/#state

So I think you should switch from
is_new_object = instance.pk is None
to
is_new_object = instance._state.adding

createinitialfieldhistory doesn't work for objects without id field

I use a uuid for a primary key on my objects, and as such call it uuid. When I go to create the initial field history, I get this error:

File "lib/python2.7/site-packages/field_history/management/commands/createinitialfieldhistory.py", line 35, in handle object_id=obj.id,
AttributeError: 'Order' object has no attribute 'id'

I fixed this by changing line 35 in createinitialfieldhistory.py to object_id=obj.pk

PostgreSQL and JSONField for serialized_data

I t's just my suggestion but I think, for PostgreSQL you shoud use fields like HStoreField or JSONField for serialized_data field.

FieldHistory.object_id causes MySQL error during migration 0001

InnoDB indexes in MySQL can only be created for columns with a max length of 767 bytes (see the third bullet under Maximums and Minimums), so the TextField that FieldHistory.object_id currently uses causes migration 0001 to throw an exception. Example output:

Operations to perform:
  Apply all migrations: field_history
Running migrations:
  Rendering model states... DONE
  Applying field_history.0001_initial...Traceback (most recent call last):
  File "project/proj/manage.py", line 8, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python2.7/site-packages/raven/contrib/django/management/__init__.py", line 41, in new_execute
    return original_func(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/usr/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 91, in __exit__
    self.execute(sql)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/base/schema.py", line 111, in execute
    cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.OperationalError: (1170, "BLOB/TEXT column 'object_id' used in key specification without a key length")

The obvious solution is to change FieldHistory.object_id to a fixed-length field, such as a CharField.

Middleware and threading

Hi!

I have a question. Your tracker uses threading.locals. Have you ever tried to test this decisiton on a production with more or less big number of users who make changes to your models at the same time?
If you use uwsgi then Django will be just forked by some number of processes (2, 4, 5, whatever you set in your uwsgi settings). So, it means that if you have 1000 online users then, e.g., and 4 forks of Django, then 250 of your 'requests' which you store in thread will go to the same locals and will overwrite each other.
To my mind, it's not serious to tell people that they should use this middleware in their project with full confidence that it will log all changes correctrly.

How does it handle migrations?

I like your approach to recording model history. I've tried using django-reversion before but it broke as I migrated my models by adding or deleting fields. I wonder, have you considered how django-field-history would handle migrations?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.