Serialized SQLAlchemy Query resulting in empty dictionaries or an error about marshmallow

[makmanalp]

🎉 🎈 👍 !!

[xYDanTheManYx]

Using sqlalchemy, flask & marshmallow.
I also seem to have an issue when a sqlalchemy query result is as below:

Query: Hosts.query.orderby(Hosts.hostname).all()

hosts
[<main.Hosts object at 0x102b710d0>, <main.Hosts object at 0x102b6ea10>]

Serializer(hosts, many=True).data
[collections.OrderedDict({}), collections.OrderedDict({})]

Serializer(hosts, only=('hostname'), many=True).data
[collections.OrderedDict({'h': None, 'o': None, 's': None, 't': None, 'n': None, 'a': None, 'm': None, 'e': None }), collections.OrderedDict({'h': None, 'o': None, 's': None, 't': None, 'n': None, 'a': None, 'm': None, 'e': None })]

[sloria]

Can you post the code for your Serializer class?

[xYDanTheManYx]

from marshmallow import Serializer

I am not using any other classes, from the docs it appears to be optional.

[sloria]

It is absolutely necessary to subclass Serializer; it is how you define the output fields for the serializer. Can you point to the section of the documentation that was misleading?

Here is a working example using your above data:

import sqlalchemy
from marshmallow import Serializer, fields, pprint

labels = ['host', 'name', 'enabled']
modules = [sqlalchemy.util._collections.KeyedTuple((u'HostA', u'Backup', 1), labels=labels), 
           sqlalchemy.util._collections.KeyedTuple((u'HostA', u'Backup', 0), labels=labels), 
           sqlalchemy.util._collections.KeyedTuple((u'HostA', u'Backup', 0), labels=labels), 
           sqlalchemy.util._collections.KeyedTuple((u'HostA', u'Backup', 1), labels=labels)
           ]


class MySerializer(Serializer):
    name = fields.String()
    kind = fields.String()
    enabled = fields.Boolean()

serialized = MySerializer(modules, many=True, only=('name', 'enabled'))
serialized.data
# {"name": "HostA", "enabled": true}
# {"name": "HostA", "enabled": false}
# {"name": "HostA", "enabled": false}
# {"name": "HostA", "enabled": true}

[xYDanTheManYx]

I have tried it like you have specified and it worked.
After rereading the docs it appears I have completely overlooked having to subclass Serializer.

Previously I had tried subclassing like this:

class MySerializer(Serializer):
class Meta:
fields = ('id', 'name', 'enabled')

But received an error when using the class:
TypeError: tuple indices must be integers, not str

As I now have it working I will keep it to how you have specified.

Thanks very much for your assistance!

[sloria]

Glad I could help.

Your issue actually brings up an issue that I had not found before: using the fields option in class Meta does not work for keyed tuples such as sqlalchemy's KeyedTuple and collections.namedtuple. Thanks for finding this! I will issue a fix soon.

[makmanalp]

Hey @sloria - I just ran into this again, with the fields & KeyedTuple. I think the issue is that it uses dir(). Here's a debugger session from that error, in marshmallow/utils.py:to_marshallable_type():

>>> obj
sqlalchemy.util._collections.KeyedTuple((3278477.5500000003, 29700, 0.04472251026943858, 0.8179699425486985, 0.4208957237122138, 0.8067728413148435, 2479, 3,  ... ))

>>> dir(obj)
Traceback (most recent call last):
File "<debugger>", line 1, in <module>
dir(obj)
TypeError: unorderable types: NoneType() < str()

>>> obj.__dict__.keys()
dict_keys(['year', 'product_id', 'import_value', None, 'department_id', 'cog', 'coi', 'export_rca', 'export_value', 'id', '_labels'])

Sort of ...

I'm using class Meta: fields=[], and still seems to break.

[makmanalp]

And oddly specifying only=() doesn't seem to fix it for me as it did for you guys.

[sloria]

Thanks for reporting @makmanalp. I will look into this soon.

[makmanalp]

@sloria no, thank /you/!