zlorf / django-synchro Goto Github PK
View Code? Open in Web Editor NEWDjango app for database data synchronization.
License: MIT License
Django app for database data synchronization.
License: MIT License
In Django 1.7 for each syncdb command I have:
The following content types are stale and need to be deleted:
synchro |
Seems it comes from options = SynchroSettings()
in models.py
I wanted just to test the library to use my app as a database repository so I set it up very basically:
Then I deployed an app on heroku cloud and copied my database from app. Then I changed few things locally. After that I tried to synchronize it via localhost:8000/synchro
and via python manage.py synchronize
The outcome is quite strange: for the browser synchro I always get the following error:
An error occured: Cannot assign "<Object name>": the current database router prevents this relation. (ValueError)
for shell command I always get the No changes since last synchronization.
communicate
๐ข
When I try to synchronize via shell or admin then I see the following error:
django.db.utils.ProgrammingError: relation "synchro_changelog" does not exist
LINE 1: ..."."app_label", "django_content_type"."model" FROM "synchro_c...
In changelog i have a date with millisecond: 2015-01-26 15:00:23.559368 and in dbsettings for last_check without: 2015-01-26 13:26:56. Then it makes an update for remote on every synchronization.
To solve this, i used:
SYNCHRO_DATETIME_INPUT_FORMATS = ('%Y-%m-%d %H:%M:%S.%f',)
class DateTimeMsValue(dbsettings.DateTimeValue):
formats_source = 'SYNCHRO_DATETIME_INPUT_FORMATS'
class SynchroSettings(dbsettings.Group):
last_check = DateTimeMsValue('Last synchronization', default=now())
Is there any better solution?
If you use
objects = natural_manager('foo', 'bar')
def natural_key(self):
return self.foo, self.bar
in a model that is part of a many to many relationship, the synchronize command will throw an exception. Here's a shell log for the statement that throws the exception.
In [2]: obj = ProductMeta.objects.get(pk=89)
In [3]: obj
Out[3]: <ProductMeta: Variabel Stroom>
In [4]: obj._meta.many_to_many
Out[4]:
[<django.db.models.fields.related.ManyToManyField: sites>,
<django.db.models.fields.related.ManyToManyField: terms>,
<django.db.models.fields.related.ManyToManyField: switchurls>,
<django.db.models.fields.related.ManyToManyField: usps>]
In [5]: for f in obj._meta.many_to_many:
...: f.value_from_object(obj)
...:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/Users/adriaant/.virtualenvs/tool/lib/python2.6/site-packages/django/core/management/commands/shell.pyc in <module>()
1 for f in obj._meta.many_to_many:
----> 2 f.value_from_object(obj)
/Users/adriaant/.virtualenvs/tool/lib/python2.6/site-packages/django/db/models/fields/related.pyc in value_from_object(self, obj)
1241 def value_from_object(self, obj):
1242 "Returns the value of this field in the given model instance."
-> 1243 return getattr(obj, self.attname).all()
1244
1245 def save_form_data(self, instance, data):
/Users/adriaant/.virtualenvs/tool/lib/python2.6/site-packages/django/db/models/fields/related.pyc in __get__(self, instance, instance_type)
817 target_field_name=self.field.m2m_reverse_field_name(),
818 reverse=False,
--> 819 through=self.field.rel.through,
820 )
821
/Users/adriaant/.virtualenvs/tool/lib/python2.6/site-packages/django/db/models/fields/related.pyc in __init__(self, model, query_field_name, instance, symmetrical, source_field_name, target_field_name, reverse, through, prefetch_cache_name)
521 source_field_name=None, target_field_name=None, reverse=False,
522 through=None, prefetch_cache_name=None):
--> 523 super(ManyRelatedManager, self).__init__()
524 self.model = model
525 self.query_field_name = query_field_name
TypeError: __init__() takes at least 2 arguments (1 given)
If I remove the natural_manager statement and the natural_key method, this will work. It looks like it's this bug:
https://code.djangoproject.com/ticket/13313#comment:9
I'm not sure what is the best way to solve this.
Creation of model with m2m fields are handled properly.
However, when model once synced is changed (exactly: its m2m fields is changed), it is not logged, hence not synced later.
For testing purposes, I want to use my local machine as LOCAL and a separate test machine as REMOTE. You recommend to also add 'synchro' in the REMOTE INSTALLED_APPS, but what should I then use as REMOTE setting in DATABASES, because the test machine cannot connect to my localhost database?
When running any management command in Django 1.7 (runserver, shell...), it crashes, apparently because models are loaded too early:
https://docs.djangoproject.com/en/1.7/ref/applications/#troubleshooting
Stacktrace:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
django.setup()
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
app_config = AppConfig.create(entry)
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/apps/config.py", line 87, in create
module = import_module(entry)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/synchro/__init__.py", line 4, in <module>
from management.commands.synchronize import call_synchronize
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/synchro/management/__init__.py", line 4, in <module>
from synchro import models
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/synchro/models.py", line 8, in <module>
import settings # in order to validate
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/synchro/settings.py", line 45, in <module>
MODELS = parse_models(getattr(settings, 'SYNCHRO_MODELS', ()))
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/synchro/settings.py", line 30, in parse_models
res.extend(get_all_models(entry))
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/synchro/settings.py", line 11, in get_all_models
return get_models(app_mod)
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/utils/lru_cache.py", line 101, in wrapper
result = user_function(*args, **kwds)
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/apps/registry.py", line 168, in get_models
self.check_models_ready()
File "/home/.virtualenvs/project/local/lib/python2.7/site-packages/django/apps/registry.py", line 131, in check_models_ready
raise AppRegistryNotReady("Models aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
I'm trying to find a workaround for this, not sure if it's possible to fix easily. The problematic code seems to be in settings.py
:
def get_all_models(app):
try:
app_mod = load_app(app) # First try full path
except ImportError:
app_mod = get_app(app) # Then try just app_label
return get_models(app_mod) # <- crashes here
1: pip install modernize
2: python-modernize -wn
details: https://portingguide.readthedocs.io/en/latest/tools.html#python-modernize
I am trying to use an django plugin at https://github.com/zlorf/django-synchro#skipping-fields
But it always say "No changes since last synchronization."
Here is settings snippet:
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'wnitest',
'dbsettings',
'synchro',
'xadmin',
'crispy_forms',
'debug_toolbar',
)
DATABASES = {
"default": {
# Ends with "postgresql_psycopg2", "mysql", "sqlite3" or "oracle".
"ENGINE": "django.db.backends.mysql",
# DB name or path to database file if using sqlite3.
"NAME": "djangotest",
# Not used with sqlite3.
"USER": "root",
# Not used with sqlite3.
"PASSWORD": "root",
# Set to empty string for localhost. Not used with sqlite3.
"HOST": "",
# Set to empty string for default. Not used with sqlite3.
"PORT": "",
},
"back": {
# Ends with "postgresql_psycopg2", "mysql", "sqlite3" or "oracle".
"ENGINE": "django.db.backends.mysql",
# DB name or path to database file if using sqlite3.
"NAME": "djangotest2",
# Not used with sqlite3.
"USER": "root",
# Not used with sqlite3.
"PASSWORD": "root",
# Set to empty string for localhost. Not used with sqlite3.
"HOST": "localhost",
# Set to empty string for default. Not used with sqlite3.
"PORT": "",
}
}
SYNCHRO_REMOTE = 'back'
SYNCHRO_DEBUG = True
So,when I open admin and add some objects,all saved successfully.
Then, issue python manage.py synchronize, always says "No changes since last synchronization."
I see there are synchro related tables:
mysql> show tables;
+----------------------------+
| Tables_in_djangotest |
+----------------------------+
| auth_group |
| auth_group_permissions |
| auth_permission |
| auth_user |
| auth_user_groups |
| auth_user_user_permissions |
| dbsettings_setting |
| django_admin_log |
| django_content_type |
| django_migrations |
| django_session |
| synchro_changelog |
| synchro_deletekey |
| synchro_reference |
| wnitest_answer |
| wnitest_question |
| xadmin_bookmark |
| xadmin_usersettings |
| xadmin_userwidget |
+----------------------------+
My setup:
django: 1.7
django-dbsettings : 0.8
Anything wrong with my configuration or synchro cannot be used to local two databases?
Thanks.
Wesley
Is django-synchro support 1.11?
Hope this project is still active developing
If you have something like:
charges = models.ManyToManyField(Price, through='SupplierCharge', blank=True, null=True, verbose_name=_('charge'))
in a model, then synchronize fails in synchro/management/commands/synchronize.py", line 99, in save_with_fks
f.save_form_data(obj, out) with:
AttributeError: Cannot set values on a ManyToManyField which specifies an intermediary model. Use energy.SupplierCharge's Manager instead.
I'm not using the natural_manager() in the models involved.
When i edit an object with m2m field through admin there is no m2m_changed signal and m2m changes are not synced.
Maybe it's better to add M2M_CHANGE action for each ADDITION and CHANGE?
if del_time is None or last_time > del_time:
ACTIONS[log.action](log.content_type, log.object_id, log)
if log.action in [ADDITION, CHANGE]:
ACTIONS[M2M_CHANGE](log.content_type, log.object_id, log)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.