umutbozkurt / django-rest-framework-mongoengine Goto Github PK

Now serialization fails with error

DBRef('item', ObjectId('540cddee421aa93370ffa049')) is not JSON serializable

The reason is in result of ListField.to_native method:

[bson.son.SON({'_cls': 'Item', '_ref': DBRef('item', ObjectId('540cddee421aa93370ffa049'))}), ...]

It seems it is related to commit 4cae11f. Reverting this commit solves issue for me, but I'm not sure for what original commit was intended.

views.py

from rest_framework import generics

from .models import User
from .serializers import UserListSerializer

class UserList(generics.ListCreateAPIView):
    queryset = User.objects.all()
    paginate_by = 100
    serializer_class = UserListSerializer


user_list = UserList.as_view()

models.py

import binascii
import os

from django.contrib.auth.models import AbstractUser
from django.db import models

class User(AbstractUser):

    token = models.CharField(name='Token', blank=True, max_length=50)

    def __unicode__(self):
        return self.username

    class Meta:
        db_table = 'users'

    @staticmethod
    def generate_token():
        return binascii.hexlify(os.urandom(20)).decode()

    def make_new_token(self):
        self.token = self.generate_token()
        return self.save()

serializers.py

from rest_framework_mongoengine.serializers import MongoEngineModelSerializer
from .models import User

class UserListSerializer(MongoEngineModelSerializer):
    class Meta:
        model = User
        depth = 2
        exclude = ('password',)

Full traceback:

Traceback (most recent call last):
  File "/home/vagrant/.virtualenvs/nonrel/src/django/django/core/handlers/base.py", line 113, in get_response
    response = callback(request, *callback_args, **callback_kwargs)
  File "/home/vagrant/.virtualenvs/nonrel/src/django/django/views/decorators/csrf.py", line 77, in wrapped_view
    return view_func(*args, **kwargs)
  File "/home/vagrant/.virtualenvs/nonrel/src/django/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework/views.py", line 403, in dispatch
    response = self.handle_exception(exc)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework/views.py", line 400, in dispatch
    response = handler(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework/generics.py", line 493, in get
    return self.list(request, *args, **kwargs)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework/mixins.py", line 98, in list
    return Response(serializer.data)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 576, in data
    self._data = self.to_native(obj)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 355, in to_native
    value = field.field_to_native(obj, field_name)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 414, in field_to_native
    return [self.to_native(item) for item in value.all()]
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework_mongoengine/serializers.py", line 203, in to_native
    dynamic_fields = self.get_dynamic_fields(obj)
  File "/home/vagrant/.virtualenvs/nonrel/local/lib/python2.7/site-packages/rest_framework_mongoengine/serializers.py", line 131, in get_dynamic_fields
    if obj is not None and obj._dynamic:
AttributeError: 'User' object has no attribute '_dynamic'

When uploading an image in ImageField, django-rest-framework returns warning "Upload a valid image. The file you uploaded was either not an image or a corrupted image." and the returned value is null. However, image.files and image.chunks can be found in mongodb. The problem is how to get URL of uploaded image correctly?

Updating to version 2.0.2, the framework fails with GeoPointField declaration:
u"<class 'mongoengine.fields.GeoPointField'> is not supported, yet. Please open a ticket regarding this issue and have it fixed asap.\nhttps://github.com/umutbozkurt/django-rest-framework-mongoengine/issues/"
With a previous version, we were able to use this field in a model or at least to use it in the declaration.
Thanks and keep the great work !!

GenericAPIView uses .lookup_url_kwarg -- this is quite clean and documented
MongoAPIView uses all query_kwarg -- this looks somewhat weird

Also, GenericAPIView.get_obejct throws http not found
MongoAPIView raises internal server error

Is there some reason to do that?

Just like all other fields do when input is inconvertable.
Surrounding frameworks would know nothing about bson.errors

class ObjectIdField(fields.Field):
    type_label = 'ObjectIdField'

    def to_representation(self, value):
        return smart_str(value)

    def to_internal_value(self, data):
        try:
            return ObjectId(data)
        except bson.errors.InvalidId as e:
            raise ValidationError(e)

drf_2_support version does not support write_only_fields

I am trying to retrieve a list of posts but for referenced users in the "likes" field I do not want some fields such as 'email' exposed. In the User model I have the field 'email' as excluded but it still shows up when looking at posts. Ontop of that, the 'likes' field should just return a list of objectID's should it not? Why is it expanding the referenced user when the depth is 1?
I have a model setup like this:

model

class User(Document):
    name = StringField(max_length=50)
    surname = StringField(max_length=20)
    username = StringField(max_length=30)
    email = EmailField(max_length=30)
    #friends = ListField(ReferenceField('self'))
    #UUID = UUIDField()

class Post(Document):
    author = ReferenceField(User)
    #blog = ReferenceField(Blog)
    text = StringField()
    likes = ListField(ReferenceField('User'))

Serializers

class UserSerializer(MongoEngineModelSerializer):
    class Meta:
        model = User
        depth = 2
        exclude = ('email',)

class PostSerializer(MongoEngineModelSerializer):
    class Meta:
        model = Post
        depth = 1

I attempted to create a new serializer to only expose certain field for the user.

Serializers

...
class LimitedUserSerializer(MongoEngineModelSerializer):
    class Meta:
        model = User
        depth = 1
        fields = ('name',)

class PostSerializer(MongoEngineModelSerializer):
    likes = LimitedUserSerializer(many=True)
    class Meta:
        model = Post
        depth = 1

This had the desired effect of not exposing too many fields but now I am unable to PUT new users in the "likes" field. I get an attribute error as understandable

POST

{
"likes": [ "54554428e138230be2ab0b25", "54554428e138230be2ab0b26" ],
"id": "54554428e138230be2ab0b25",
"author": "54554428e138230be2ab0b25",
"text": "This is a new post"
}

Error

AttributeError at /posts/54554428e138230be2ab0b25/
'unicode' object has no attribute 'keys'
Request Method: POST
Request URL:    http://127.0.0.1:8000/posts/54554428e138230be2ab0b25/
Django Version: 1.7.1
Exception Type: AttributeError
Exception Value:    
'unicode' object has no attribute 'keys'

/home/vagrant/projects/test_project/rest_framework_mongoengine/serializers.py in from_native
            for key in data.keys(): ...
Traceback:
File "/usr/lib64/python2.7/site-packages/django/core/handlers/base.py" in get_response
  111.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/lib64/python2.7/site-packages/django/views/decorators/csrf.py" in wrapped_view
  57.         return view_func(*args, **kwargs)
File "/usr/lib64/python2.7/site-packages/django/views/generic/base.py" in view
  69.             return self.dispatch(request, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
  403.             response = self.handle_exception(exc)
File "/usr/lib/python2.7/site-packages/rest_framework/views.py" in dispatch
  400.             response = handler(request, *args, **kwargs)
File "/home/vagrant/projects/test_project/rest_framework_mongoengine/generics.py" in put
  144.         return self.update(request, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/rest_framework/mixins.py" in update
  122.         if not serializer.is_valid():
File "/usr/lib/python2.7/site-packages/rest_framework/serializers.py" in is_valid
  554.         return not self.errors
File "/usr/lib/python2.7/site-packages/rest_framework/serializers.py" in errors
  546.                 ret = self.from_native(data, files)
File "/home/vagrant/projects/test_project/rest_framework_mongoengine/serializers.py" in from_native
  228.             attrs = self.restore_fields(data, files)
File "/usr/lib/python2.7/site-packages/rest_framework/serializers.py" in restore_fields
  287.                 field.field_from_native(data, files, field_name, reverted_data)
File "/usr/lib/python2.7/site-packages/rest_framework/serializers.py" in field_from_native
  473.                 if serializer.is_valid():
File "/usr/lib/python2.7/site-packages/rest_framework/serializers.py" in is_valid
  554.         return not self.errors
File "/usr/lib/python2.7/site-packages/rest_framework/serializers.py" in errors
  536.                         ret.append(self.from_native(item, None))
File "/home/vagrant/projects/test_project/rest_framework_mongoengine/serializers.py" in from_native
  229.             for key in data.keys():

Exception Type: AttributeError at /posts/54554428e138230be2ab0b25/
Exception Value: 'unicode' object has no attribute 'keys'

Am I modeling the data wrong or is this a bug? Basically all I am trying to achieve is not exposing referenced fields that may be sensitive such as a users phone number or similar

Hi. Thanks for developing this app. I"m having m first play with django+mongo+DWR.

I looking at how to filter results for the ListView. When I used a db backend, the GenericAPIViews work really well with the DjangoFilterBackend. This however relies on django-filter, which at a cursory glance is closely tied to django.db.models.

I've achieved filtering using the mongonengine_generics by overriding get_queryset(). I was just wondering if there was a clean and simple way to add filter_fields to the mongonengine_generics Views? anted to make sure i wasn't missing anything.

Thanks

Hi, try to update (put,patch) a record raise error 500 (Mod on _id not allowed)

for some reason mongoengine detect the _id field as "changed"

this is my relevant code :

serializer.py

from models import CustomUser
from rest_framework_mongoengine.serializers import MongoEngineModelSerializer
from django.contrib.auth.hashers import make_password


class CustomUserSerializer(MongoEngineModelSerializer):
    class Meta:
        model = CustomUser
        exclude = ['user_permissions']
        read_only_fields = ['id']
        write_only_fields = ('password',)

    def restore_object(self, attrs, instance=None):
        # call set_password on user object. Without this
        # the password will be stored in plain text.
        user = super(CustomUserSerializer, self).restore_object(attrs, instance)
        user.password = make_password(attrs['password'])
        return user

model.py

from mongoengine import ListField
from mongoengine.django.auth import User

class CustomUser(User):
    USERNAME_FIELD = 'username'
    REQUIRED_FIELDS = ['email']
    tags = ListField(default='')

view.py

from models import CustomUser
from serializer import CustomUserSerializer
from rest_framework_mongoengine import viewsets

class UserListViewSet(viewsets.ModelViewSet):
    serializer_class = CustomUserSerializer
    queryset = CustomUser.objects.all()
    model = CustomUser

urls.py

from django.conf.urls import patterns, url, include
from rest_framework_mongoengine.routers import MongoDefaultRouter
from users import views


router = MongoDefaultRouter()
router.register(r'users', views.UserListViewSet, 'list')

urlpatterns = patterns('',
    url(r'^', include(router.urls)),
    url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework'))
)

bye

When attempting to declare a property that should be serialized as in the following scenario:

http://stackoverflow.com/questions/17066074/modelserializer-using-model-property

There is an exception thrown in serializers.py line 212.
if issubclass(obj._data[key].class, mongoengine.Document) or isinstance(obj._data[key], DBRef):

The issue appears to be that to_native() is expecting all items in 'fields' to be field types, not a mix of fields and properties. When adding a try/except around the statement, the remainder of the function handles this case correctly

in new version of rest_framework, compat.py doesn't contains 'get_concrete_model' function
serailizer works well in this module of version 1.3.2, but doesn't work with version which throwing Import Error.

u"<class 'mongoengine.fields.BinaryField'> is not supported, yet

I have a lot of fun with your framework, guys!
Maybe you should test yout serializer against all mongoengine fields?

As workaround, I've mapped PointField to MongoDocumentField,
but it probably should do some correct validation.

The rest_framework.serializers.ModelSerializer intentionally uses ordered dicts for fields to preserve declaration order. Mongoengine also supports field declaration order and it should be maintained.

In MongoEngineModelSerializer.get_default_fields:

fields += [getattr(opts, field) for field in opts._fields]

Should be:

fields += [getattr(opts, field) for field in cls._fields_ordered]

In MongoEngineModelSerializer.to_native:

ret = self._dict_class()
ret.fields = self._dict_class()

#Dynamic Document Support
dynamic_fields = self.get_dynamic_fields(obj)
all_fields = dict(dynamic_fields, **self.fields)

Should be:

all_fields = self._dict_class()
all_fields.update(self.get_dynamic_fields(obj))
all_fields.update(self.fields)

ListField is now implemented in drf-3.0
And anyway it should not be DocumentField, because can be used outside of DicumentSerializer context.

Hello,
I looks like a really important bug was fixed on master, but it exists in 1.2 (that is accessible on pypi )
It has something to do with 'required' parameter being passed to ObjectIdField (probably at some point mongoengine upgrade introduced this incomparability)

I love this project, but I almost abandoned it when run into that problem...
Even currently in my requirements.txt I have to put a link to the git repo instead of the actual version.
Looks like a good time for 1.3 (or 1.2.x)

Im not sure if this is related to the bug with the depth field not working but I'm having the following issue.

My model looks like this:
class UserProfile(Document):
friends = ListField(ReferenceField('self'))

If user 1 and user 2 are "friends" (ie: they both have eachother's uid saveed in the friends list) then if i browse to user1's detail page, it creates an endless loop because it then tries to show user2's information, then it tries to show user2's friends..and so forth. I'm fairly sure this is a bug

DRF serializer allows us to create WritableFields for custom parsing/serializing/validation.
Fields lists and dicts should not be exceptions.
Proper implementation of list and dict fields is at: https://github.com/estebistec/drf-compound-fields/

Example:

class FooSerializer(rest_framework_mongoengine.serializers.MongoEngineModelSerializer):
  class Meta:
    model = Foo
  phones = compfields.ListField(source='phones', item_field=PhoneField())

class PhoneField(rest_framework.fields.CharField):
  def from_native(self, value): 
    # strip non digits
  def validate(self, value):
    # count digits

N.B. 
In DRF3 WritableField is removed, and they have their own ListField.
They also recomend to remove any db-model-level validation and perform it on serializer level.

I have a model that looks like this:

class UserProfile(Document):
geo_location = GeoPointField()

Im getting the following error:
Traceback:
File "/home/vagrant/envs/leo/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response

1.                 response = wrapped_callback(request, _callback_args, *_callback_kwargs)
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/django/views/decorators/csrf.py" in wrapped_view

2.     return view_func(_args, *_kwargs)
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/django/views/generic/base.py" in view

3.         return self.dispatch(request, _args, *_kwargs)
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/views.py" in dispatch

4.         response = self.handle_exception(exc)
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/views.py" in dispatch

5.         response = handler(request, _args, *_kwargs)
   

   File "/home/vagrant/projects/leo_project/leo/leo_app/api/views.py" in get

6.     data = {'nearby_friends': serialized_users.data, 'open_events': serialized_events.data}
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/serializers.py" in data

7.     ret = super(ListSerializer, self).data
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/serializers.py" in data

8.             self._data = self.to_representation(self.instance)
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/serializers.py" in to_representation

9.         self.child.to_representation(item) for item in iterable
   

   File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/serializers.py" in to_representation

10.             ret[field.field_name] = field.to_representation(attribute)
    

    File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/serializers.py" in to_representation

11.         self.child.to_representation(item) for item in iterable
    

    File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/serializers.py" in to_representation

12.     fields = [field for field in self.fields.values() if not field.write_only]
    

    File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework/serializers.py" in fields

13.         for key, value in self.get_fields().items():
    

    File "/home/vagrant/envs/leo/lib/python2.7/site-packages/rest_framework_mongoengine/serializers.py" in get_fields

14.                                type(model_field))
    

Exception Type: KeyError at /api/home/
Exception Value: u"<class 'mongoengine.fields.GeoPointField'> is not supported, yet. Please open a ticket regarding this issue and have it fixed asap.\nhttps://github.com/umutbozkurt/django-rest-framework-mongoengine/issues/"

Hi, guys
Actually when using rest_framework_mongoengine.serializers.DocumentSerializer I've faced with that problem.
For me getting that fixed is quite important.
File "/home/vpaslav/.virtualenvs/project/lib/python2.7/site-packages/rest_framework_mongoengine/serializers.py", line 159, in is_valid valid = super(DocumentSerializer, self).is_valid(raise_exception=raise_exception) File "/home/vpaslav/.virtualenvs/project/lib/python2.7/site-packages/rest_framework/serializers.py", line 158, in is_valid self._validated_data = self.run_validation(self._initial_data) File "/home/vpaslav/.virtualenvs/project/lib/python2.7/site-packages/rest_framework/serializers.py", line 313, in run_validation value = self.to_internal_value(data) File "/home/vpaslav/.virtualenvs/project/lib/python2.7/site-packages/rest_framework/serializers.py", line 352, in to_internal_value field for field in self.fields.values() File "/home/vpaslav/.virtualenvs/project/lib/python2.7/site-packages/rest_framework/serializers.py", line 245, in fields for key, value in self.get_fields().items(): File "/home/vpaslav/.virtualenvs/project/lib/python2.7/site-packages/rest_framework_mongoengine/serializers.py", line 255, in get_fields type(model_field)) KeyError: u"<class 'mongoengine.fields.PointField'> is not supported, yet. Please open a ticket regarding this issue and have it fixed asap.\nhttps://github.com/umutbozkurt/django-rest-framework-mongoengine/issues/"
Thanks in advance,
Vlad

This is probably not exactly drf-mongo issue, but it is related. I have not digged enough deep to the nature of the bug.
Defining MongoAPIView.queryset is not absolutely required to work, but it is documented practice.
When it's defined, collection object is created and referenced/cached somewhere before testrunner runs.
Reconnecting to mongo leaves the collection intact and all testing modifications go into production database.

I've made gist to reproduce and explore:
https://gist.github.com/qwiglydee/67a68638ee9373c2982b

Short story:

Sometimes I can get 'stale' results when querying a database. Database had correct values. REST interface showed previous value.

Long story:

One of my unit tests was reporting the wrong value from an endpoint I was doing list operations on. I would do a POST (successfully completed) with a GET immediately after it.

My work-around:

I noticed that get_queryset was not defined in GenericAPIView. In rest_framework/generics.py the GenericAPIView checks that the query set is of type QuerySet (which is not the case).

Root cause analysis lead me here:

django-rest-framework-mongoengine/rest_framework_mongoengine/generics.py

Better fix than this (major hack):

class GenericAPIView(drf_generics.GenericAPIView):

    def get_queryset(self):

        queryset = self.queryset
        self.logger.debug('refreshing query set')
        queryset = queryset.all()

        return queryset

In restframework serializer augument felds with attributs label, help_text, type_label
They can be used to generate documentation.
The django rest swagger ignores them, but anyway it is good stuff.

My fix in wrapper:

    field = super(MongoModelSerializer,self).get_field(model_field)
    field.label = model_field.verbose_name
    field.help_text = model_field.help_text
    field.type_label = self.make_type_label(model_field)

def make_type_label(self, model_field):
    if isinstance(model_field, mongoengine.ListField):
        return "list(%s)" % self.make_type_label(model_field.field)
    elif isinstance(model_field, mongoengine.EmbeddedDocumentField):
        return "embedded(%s)" % model_field.document_type_obj.__name__
    elif isinstance(model_field, mongoengine.ReferenceField):
        return "reference(%s)" % model_field.document_type_obj.__name__
    else:
        return model_field.__class__.__name__.lower().replace('field','')

Native value of type uuid.UUID is not json-serializable by rest_framework jsonencoder.

The serializer should probably map it to string field, or some custom Field with proper to_native/from_native

When attempting to serialize my Document I get the error ObjectId(...) is not JSON serializable

Models

class ProductType(Document):
    name = StringField(required=True)
    slug = StringField(required=True)

    def __unicode__(self):
        return self.name


class ProductCategory(Document):
    name = StringField(required=True)
    slug = StringField(required=True)
    description = StringField()
    image = StringField()

    def __unicode__(self):
        return self.name


class ProductDocument(EmbeddedDocument):
    title = StringField(required=True)
    slug = StringField(required=True)
    description = StringField()
    type = ReferenceField(ProductType)
    categories = ListField(ReferenceField(ProductCategory))
    attribute_values = DictField()

    def __unicode__(self):
        return self.title


class Product(Document):
    draft = EmbeddedDocumentField(ProductDocument)
    published = EmbeddedDocumentField(ProductDocument)
    versions = DictField()

    def __unicode__(self):
        return self.draft.title

    def __init__(self, *args, **kwargs):
        super(Product, self).__init__(*args, **kwargs)

        if not self.draft:
            self.draft = ProductDocument()

Serializers

class ProductSerializer(MongoEngineModelSerializer):

    class Meta:
        model = Product
        depth = 1

Views

class ProductView(ListAPIView):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer

The above error is caused by ProductDocument.type.

I've tried the sample app included with this project, and this seems to work ok. Am I structuring my document in a poor way, or is this something that can be fixed?

GenericAPIView uses django.shortcuts.get_object_or_404
MongoAPIView should use mongoengine.django.shortcuts.get_document_or_404

My version of the view that follows GenericAPIView behaviour (not welltested):

def get_object_or_404(queryset, *filter_args, **filter_kwargs):
    """ overriden to use proper _or_404 """
    try:
        return get_document_or_404(queryset, *filter_args, **filter_kwargs)
    except (TypeError, ValueError):
        raise Http404

class MongoGenericAPIView(GenericAPIView):
    """ GenericAPIView, adapted for mongoengine """
    def get_queryset(self):
        "overriden to use .clone and .objects"
        if self.queryset is not None:
            return self.queryset.clone()

        if self.model is not None:
            return self.model.objects.all()

        raise ImproperlyConfigured("'%s' must define 'queryset' or 'model'"
                                    % self.__class__.__name__)

    def get_object(self, queryset=None):
        "overriden to catch notfound"

        # Determine the base queryset to use.
        if queryset is None:
            queryset = self.filter_queryset(self.get_queryset())
        else:
            pass  # Deprecation warning

        # Perform the lookup filtering.
        lookup_url_kwarg = self.lookup_url_kwarg or self.lookup_field
        lookup = self.kwargs.get(lookup_url_kwarg, None)

        if lookup is not None:
            filter_kwargs = {self.lookup_field: lookup}
        else:
            raise ImproperlyConfigured(
                'Expected view %s to be called with a URL keyword argument '
                'named "%s". Fix your URL conf, or set the `.lookup_field` '
                'attribute on the view correctly.' %
                (self.__class__.__name__, self.lookup_field)
            )

        obj = get_object_or_404(queryset, **filter_kwargs)

        # May raise a permission denied
        self.check_object_permissions(self.request, obj)

        return obj

For both serializers and documentserializers.

Add-on implementations, like drf-compund-fields cannot be used without patching DocumentSerializing.field_mapping.

I have implemented it in the similar way as rest_framework.fields.ListField and can provide patch/pullrequest.

A question still remains whether children/subfields definition should be strict and unexpected items should be reported, or undeclared items should go as strings without errors.
My guess is that for strict subfields embedded documents should be used.

Getting error message on version 1.5.3

DBRef('user_profile', ObjectId('542e37c6e138230c28cf44d2')) is not JSON serializable

Serializer
class UserProfileSerializer(MongoEngineModelSerializer):
class Meta:
model = UserProfile
fields = ('email', 'first_name', 'friends', 'city')

Model
class UserProfile(Document):
email = StringField()
first_name = StringField()
city = StringField()
friends = ListField(ReferenceField('self'))

def __unicode__(self):
        return '%s' % (self.id)

Database
user1
{
"_id" : ObjectId("542e37aae138230c28cf44d2"),
"first_name" : "craig",
"friends" : [
ObjectId("542e37c6e138230c28cf44d3")
],
"city" : "",
"email" : "",
}

user2
{
"_id" : ObjectId("542e37c6e138230c28cf44d3"),
"first_name" : "george",
"friends" : [
ObjectId("542e37c6e138230c28cf44d2")
],
"email" : "'[email protected]'",
"city" : "",
}

Tried
I tried to put this in my serializer

friends = serializers.RelatedField(many=True)

which gets rid of the TypeError but then then the friends list looks empty as shown below.
"friends": [
[
"email",
"first_name",
"city",
"friends",
]
],

I also tried different depths but didnt make a difference.

You have closed related issue, but no released version exists.
(1.4.3 at github releases and 1.5.4 at pipy are incompatible)

Hi @umutbozkurt
First of all, thank you, and other contributors, for this gem.

I just want to double-check that I am doing things right in my project. I noticed that Boolean are serialized differently in Document and EmbeddedDocument.

Model:

class BlogExtension(EmbeddedDocument):
    extra_field = StringField()
    boolean_field = BooleanField(default=False)


class Blog(DynamicDocument):
    title = StringField(max_length=30)
    isActive = BooleanField(default=True)
    extensions = ListField(EmbeddedDocumentField(BlogExtension))

Serializer:

class BlogSerializer(MongoEngineModelSerializer):
    class Meta:
        model = Blog
        depth = 2 

Response:

[
    {
        "id": "53fd039750656841b2d08ee4",
        "title": "foo",
        "isActive": true,
        "extensions": [
            {
                "extra_field": "bar",
                "boolean_field": "True"
            }
        ]
    }
]

Please note the difference between isActive and boolean_field
It causes some discrepancies in UI which is based on these attributes.

Was it done intentionally?
Thanks

Problem description
When there is an embedded serializer field defined in the root document serializer, DRF's BaseSerializer will throw a "Cannot call .is_valid() as no data= keyword argument waspassed when instantiating the serializer instance." AssertionError in is_valid method, create use case, based on the following assertion:

        assert hasattr(self, 'initial_data'), (
            'Cannot call `.is_valid()` as no `data=` keyword argument was'
            'passed when instantiating the serializer instance.'
        )

The first that comes to mind is that the following code in DocumentSerializer is failing to set the required property:

    def is_valid(self, raise_exception=False):
        valid = super(DocumentSerializer, self).is_valid(raise_exception=raise_exception)
        for embedded_field in self.embedded_document_serializer_fields:
            embedded_field._initial_data = self.validated_data.pop(embedded_field.field_name, serializers.empty)
            valid &= embedded_field.is_valid(raise_exception=raise_exception)
        return valid

Changing _initial_data to initial_data seems to fix the issue (looks like that only initial_data is used by BaseSerializer in DRF3), though I'm not sure of the consequences...

Test case:

# models:

class EmbeddedTest(EmbeddedDocument):
    int_field = IntField()

class Test(Document):
    string_field = StringField()
    dynamic_field = EmbeddedDocumentField(EmbeddedTest)

# serializers:

class EmbeddedTestSerializer(EmbeddedDocumentSerializer):

    class Meta:
        model = EmbeddedTest
        depth = 1


class TestSerializer(DocumentSerializer):
    dynamic_field = EmbeddedTestSerializer(many=False)

    class Meta:
        model = Test
        depth = 1

Serializer's depth argument needs to be fixed since it does not count levels of json output

Looks like mongoengine's FileField doesn't have this attribute. So are EmailField and URLField. This is needed for DRF's FileInput widget though. (ref line 340-342 in serializers.py)

DocumentSerializer lacks recognition of ChoiceField, because get_field_kwargs misses checking for model_field.choices

@@ -331,6 +331,12 @@
             kwargs['required'] = False
             kwargs['default'] = model_field.default

+        if model_field.choices:
+            # If this model field contains choices, then return early.
+            # Further keyword arguments are not valid.
+            kwargs['choices'] = model_field.choices
+            return kwargs
+
         if model_field.__class__ == models.TextField:
             kwargs['widget'] = widgets.Textarea

Am trying to update my records using django rest mongoengine RetrieveUpdateDestroyAPIView However I get a 500 error code with 'Could not save document (Mod on _id not allowed)'

DRF advices to use 3rd party project django-filters and has adapter for it.

I'm going to implement the same stuff for mongoengine and integrate it with DRFME.
Will such code suitable to join this project, or should I start separate?

This issue isn't relevant anymore. The problem was caused because of the fact that the shell allow reimporting. Hence the normal DRF was used instead of the one for mongoengine. After completely restarting the shell everything worked fine.

I'm using Django 1.6 with mongoengine and I get an attribute error.

models.py:

class Post(Document): 
    title = StringField(max_length=120, required=True) 
    author =   ReferenceField(User, reverse_delete_rule=CASCADE) 
    created = DateTimeField()        
    #tags = ListField(StringField(max_length=30)) 
    tags = ListField(ReferenceField(Tag)) 
    comments = ListField(EmbeddedDocumentField(Comment))

    meta = {'allow_inheritance': True, 'ordering': ['-created']}

serializers.py:

from rest_framework_mongoengine import serializers 
from .models import Post 
from mongoengine.django.auth import User

class UserSerializer(serializers.DocumentSerializer):
    class Meta:
        model = User
        fields = ('username', 'first_name', 'last_name')

class PostSerializer(serializers.DocumentSerializer):
    class Meta:
        model = Post

Result of python manage.py shell:

PostSerializer(posts)
Traceback (most recent call last):
File "", line 1, in
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 440, in repr
return unicode_to_repr(representation.serializer_repr(self, indent=1))
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/utils/representation.py", line 75, in serializer_repr
fields = serializer.fields
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 312, in fields
for key, value in self.get_fields().items():
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 883, in get_fields
info = model_meta.get_field_info(model)
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/utils/model_meta.py", line 63, in get_field_info
opts = model._meta.concrete_model._meta
AttributeError: 'MetaDict' object has no attribute 'concrete_model'
PostSerializer(posts).data
Traceback (most recent call last):
File "", line 1, in
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 463, in data
ret = super(Serializer, self).data
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 212, in data
self._data = self.to_representation(self.instance)
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 419, in to_representation
fields = [field for field in self.fields.values() if not field.write_only]
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 312, in fields
for key, value in self.get_fields().items():
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/serializers.py", line 883, in get_fields
info = model_meta.get_field_info(model)
File "/home/arjan/PythonDevelopment/python2-django1.6/local/lib/python2.7/site-packages/rest_framework/utils/model_meta.py", line 63, in get_field_info
opts = model._meta.concrete_model._meta
AttributeError: 'MetaDict' object has no attribute 'concrete_model'

It's not only matter of rigid development, but also a compatibility check for users.
I'm forking repo to make the stuff.

Hi umutbozkurt,

I am new to the django rest framework with mongoengine. Please help me with below error.

When a serializer is passed a data keyword argument you must call .is_valid() before attempting to access the serialized .data representation.
You should either call .is_valid() first, or access .initial_data instead.

It looks like the latest version breaks in Python 3.x due to an invalid print statement. When installing, the following error occurs:

      File "/home/mike/virtualenvs/influenster3/lib/python3.4/site-packages/rest_framework_mongoengine/fields.py", line 115
        print obj
                ^
    SyntaxError: invalid syntax

And when running an app, the following error occurs:

  File "/vagrant/source/influenster3/.virtualenv/lib/python3.4/site-packages/rest_framework_mongoengine/serializers.py", line 14, in <module>
    from fields import ReferenceField, ListField, EmbeddedDocumentField, DynamicField
ImportError: No module named 'fields'

Since DRF 3.0+ doesn't have Writable Field and ModelSerializerOptions any more, this repo doesn't support mongodb models currently. Any plan to upgrade it to a version which bases on DRF 3? Thank you.

I get the same error on a ListField. Is there a meta option that is necessary for EmbeddedDocumentField? I tried related_model_validations, but that didn't work.

The fields in question are pulled from the database without any issue and calling .to_mongo() on them gives the correct values. I tried to use transform_ to manually serialize the field, but I get the same exception.

Here is the code that produces the exception:

class EmbeddedDescriptionsReference(EmbeddedDocument):
    long_desc = StringField()
    short_desc = StringField()

    meta = {'allow_inheritance': False}


class Products(DynamicDocument):
    descriptions = EmbeddedDocumentField(EmbeddedDescriptionsReference)

    meta = {'collection': 'products'}


class ProductsSerializer(rest_framework_mongoengine.serializers.MongoEngineModelSerializer):
    class Meta:
        model = Products
        fields = ('descriptions',)

This results in the following exception

  File ".../python3.4/site-packages/rest_framework/serializers.py", line 573, in data
    self._data = self.to_native(obj)
  File ".../python3.4/site-packages/rest_framework_mongoengine/serializers.py", line 210, in to_native
    value = field.field_to_native(obj, field_name)
  File ".../python3.4/site-packages/rest_framework/fields.py", line 417, in field_to_native
    value = self.model_field._get_val_from_obj(obj)
AttributeError: 'EmbeddedDocumentField' object has no attribute '_get_val_from_obj'

Hi, which version of djangorestframework is this project based on? I'm getting an error 'SomeSerializer' object has no attribute 'augment_field'. When I looked it up, it seems MongoEngineModelSerializer expects an older version of djangorestframework. Is that right or am I doing something wrong?

Value of None is passed to EmbeddedDocumentField.to_native

I guess it should checked for None and return None, and serializer to_native should just skip missing fields.

https://github.com/umutbozkurt/django-rest-framework-mongoengine/blob/master/rest_framework_mongoengine/serializers.py#L3-L4

I think it should be catching django.core.exceptions.ValidationError only but I'm not 100% sure. I ran into some problems when I was overriding MongoEngineModelSerializer methods and it was looking for attributes not present on mongoengine.errors.ValidationError.

MongoDocumentField.transform_object does not check for None and returns unicode(obj)

Hi,

are there any plans to support DRF 3.0?
If so, do you already have a rough timeline? And/or any contribution advises?

Thanks,
-Manuel