Field value incorrect with many=True option. about marshmallow

[sloria]

Odd indeed. Do you see the correct output when you explicitly specify the field type for contact_id?

class EventSerializer(Serializer):
    action = fields.Method('action_to_text')
    type = fields.Method('type_to_text')
    event_id = fields.Integer(attribute='id')
    contact_id = fields.Integer()
    # ...

[ScottSturdivant]

Unfortunately, no, the second element still has 0 instead of None even when explicitly specifying the field type.

[sloria]

What type of object is an event?

[ScottSturdivant]

Hi there,

Thanks for your patience, I was able to come up with a simple example that demonstrates the problem.

class Foo(object):
    def __init__(self, id, other):
        self.id = id
        self.other_id = other

 class FooSerializer(Serializer):
     class Meta:
         fields = ['id', 'other_id']

Serialized singularly, with 'other_id' not set:

>>> FooSerializer(Foo(1,None)).data
OrderedDict([('id', 1), ('other_id', None)])

When multiple, with 'other_id' set in one, but not in another:

>>> lots_of_foo = [Foo(1,1), Foo(1,None)]
>>> FooSerializer(lots_of_foo, many=True).data
[OrderedDict([('id', 1), ('other_id', 1)]), OrderedDict([('id', 1), ('other_id', 0)])]

For what it's worth, changing the order (putting the instance with None in it first), results in correct serialization:

>>> lots_of_foo = [Foo(1,None), Foo(1,1), Foo(2,None)]
>>> FooSerializer(lots_of_foo, many=True).data
[OrderedDict([('id', 1), ('other_id', None)]), OrderedDict([('id', 1), ('other_id', 1)]), OrderedDict([('id', 2), ('other_id', None)])]

Hope that you're able to reproduce and that it helps track it down!

Thanks!

[ScottSturdivant]

Sorry, here's an update with an even more simple example:

class Foo(object):
    def __init__(self, id):
        self.id = id

class FooSerializer(Serializer):
    class Meta:
        fields =['id']

 >>>  FooSerializer([Foo(None), Foo(1)], many=True).data
 [OrderedDict([('id', None)]), OrderedDict([('id', 1)])]

 >>> FooSerializer([Foo(1), Foo(None)], many=True).data
 [OrderedDict([('id', 1)]), OrderedDict([('id', 0)])]

[sloria]

Oh, I see now. When you use Meta.fields option without explicitly declaring a field on the class, there is no way for the Serializer to "know" what Field class to use for serializing id until you serialize the first object in the list.

When you use implicit field creation (i.e. Meta.fields with no explicit field declaration) together with many=True, the Serializer will inspect the first element of the list to serialize and choose which Field classes to use according to this mapping: https://github.com/sloria/marshmallow/blob/4e14f192ac478f0f0a46c671a53dc55fc5831c41/marshmallow/serializer.py#L113-127 .

Because the first Foo in the list has an id of None, the Serializer defaults to using the Raw field to format id.

The fix here is to use explicit field declaration for the id field.

from marshmallow import Serializer, fields

class Foo(object):
    def __init__(self, id):
        self.id = id

class FooSerializer(Serializer):
    id = fields.Integer()
    class Meta:
        fields =['id']

FooSerializer([Foo(None), Foo(1)], many=True).data
# [OrderedDict([('id', 0)]), OrderedDict([('id', 1)])]

Hope that helps!

[ScottSturdivant]

Hey, sorry for being a bit slow here, but even in that case, it's not giving the result I might expect (that the value for 'id' in the first element is None, not 0.

My real class looks like:

class Event(db.Model):
    id = db.Column(db.Integer, nullable=True)

Since I didn't declare an explicit field type, and because of the ordering of my data, it chose a 'Raw' field type. But it really is an Integer, and is nullable, and instead of returning 0, I'd like it to just return None.

Should I just make my own NullableInteger field type?

class NullableInteger(fields.Raw):
    def format(self, value):
        if value is None:
            return value
        return int(value)

class FooSerializer(Serializer):
    id = NullableInteger()
    class Meta:
        fields =['id']

FooSerializer([Foo(None), Foo(1)], many=True).data
# [OrderedDict([('id', None)]), OrderedDict([('id', 1)])]

Thanks...going ahead and closing this issue since it's definitely just my misunderstanding!

[sloria]

You should just be able to set default=None if you want None to serialize to None.

class FooSerializer(Serializer):
    id = fields.Integer(default=None)
    class Meta:
        fields =['id']

[ScottSturdivant]

Indeed, that works great. Sorry for all of the noise!

[sloria]

No need to apologize. Happy I could help!