klen / marshmallow-peewee Goto Github PK
View Code? Open in Web Editor NEWPeewee ORM integration with the marshmallow (de)serialization library.
Peewee ORM integration with the marshmallow (de)serialization library.
In marshmallow, there's a quick way to filter which fields to show in the dump, via "only" option (http://marshmallow.readthedocs.io/en/latest/quickstart.html#filtering-output).
But this is ignored in the ModelSchema, even when the base options are being parsed and it should be recognized and used.
Hi. I think marshmallow-peewee may provide option that serialize (and deserialize) ForeignKey fields with _id
suffix.
On API we have:
{
"id": "1",
"title": "The Chronicles of Narnia",
"author": "1"
}
For me is not clear if "1" is author pseudonym or id of another model named author.
{
"id": 1,
"title": "The Chronicles of Narnia",
"author_id": 1
}
I'll be trying to hack on my own, but I'm not sure if you want this feature as part of package.
Would it be correct to use the type of field it refers to for a foreign key field?
class ModelConverter(object):
...
def convert_ForeignKeyField(self, field, **params):
return self.convert_default(field.rel_field, **params)
For creating new instance from data ID field should not be mandatory.
[dell] ~/projects/PHD $ flask shell
Python 3.6.5 (default, Apr 12 2018, 22:45:43)
[GCC 7.3.1 20180312] on linux
App: app.core [debug]
Instance: /home/ak04nv/projects/PHD/instance
>>> from app.samples.schema import SampleSchema
>>> obj, err = SampleSchema().load({})
>>> err
{'id': ['Missing data for required field.'], 'name': ['Missing data for required field.']}
>>>
It was on 1.2.1 version
If I run the example from the readme I get the following json:
{
'active': True,
'rating': 0,
'role': '7',
'name': 'Mike',
'title': None,
'id': '7',
'created': '2018-12-13T17:37:30.956255+00:00'
}
In the readme the keys role
and id
are integers, but in fact they get converted to strings. It would be nice if they where integers.
Marshmallow from version 3.0.0 when decorators such as validate_schema is used, always pass many and partial arguments. Can you add or pass them in ModelSchema.make_instance ?
From fields.py
def _deserialize(self, value, attr, data, **kwargs):
if isinstance(value, str):
return int(value)
return super(Related, self)._deserialize(value, attr, data)
But in my models.py, my primary key is a StringField, so this breaks.
A fix would be to either look at the field type of the other ModelSchema it's "Related" to, or at least let the user pass in a parameter to serialize it to.
Currently marshmallow-peewee
supports marshmallow<=3.0.0rc5
, but not higher release candidates.
As a consumer, I would like this library to support the latest release of Marshmallow once a stable version is published.
The latest release now is version 3.0.0rc9.
Here is a pending PR to address what was removed from Marshmallow, in order to support version 2 and 3:
#78
1
{'active': True,
'created': '2022-06-02T21:04:07.061651',
'id': '1',
'name': 'Mike',
'rating': 0,
'role': '1',
'title': None}
Traceback (most recent call last):
File "/Users/cforadas/PycharmProjects/flask_blueprint_testing/app.py", line 49, in <module>
result = UserSchema().load(result)
File "/Users/cforadas/PycharmProjects/flask_blueprint_testing/venv/lib/python3.10/site-packages/marshmallow_peewee/schema.py", line 90, in load
return super(ModelSchema, self).load(data, *args, **kwargs)
File "/Users/cforadas/PycharmProjects/flask_blueprint_testing/venv/lib/python3.10/site-packages/marshmallow/schema.py", line 722, in load
return self._do_load(
File "/Users/cforadas/PycharmProjects/flask_blueprint_testing/venv/lib/python3.10/site-packages/marshmallow/schema.py", line 909, in _do_load
raise exc
marshmallow.exceptions.ValidationError: {'id': ['Unknown field.']}
Process finished with exit code 1
'3.10.4 (v3.10.4:9d38120e33, Mar 23 2022, 17:29:05) [Clang 13.0.0 (clang-1300.0.29.30)]'
marshmallow==3.16.0
Marshmallow-Peewee==3.2.1
peewee==3.14.10
1. import pprint
2.
3. import peewee as pw
4. from peewee import SqliteDatabase
5. from marshmallow_peewee import ModelSchema
6. from marshmallow_peewee import Related
7. import datetime as dt
8.
9. db = SqliteDatabase(':memory:')
10.
11.
12. class Role(pw.Model):
13. name = pw.CharField(255, default='user')
14.
15. class Meta:
16. database = db # This model uses the "people.db" database.
17.
18.
19. class User(pw.Model):
20. created = pw.DateTimeField(default=dt.datetime.now())
21. name = pw.CharField(255)
22. title = pw.CharField(127, null=True)
23. active = pw.BooleanField(default=True)
24. rating = pw.IntegerField(default=0)
25. role = pw.ForeignKeyField(Role)
26.
27. class Meta:
28. database = db # This model uses the "people.db" database.
29.
30.
31. class UserSchema(ModelSchema):
32. class Meta:
33. model = User
34.
35.
36. # create the tables
37. db.create_tables([Role, User])
38.
39. # create role
40. role = Role.create()
41. print(role)
42.
43. # create a user
44. user = User.create(name='Mike', role=role.id)
45.
46. result = UserSchema().dump(user)
47. pprint.pprint(result)
48.
49. result = UserSchema().load(result)
50. assert isinstance(result, User)
51. assert result.name == 'Mike'
52.
53.
54. class UserSchema(ModelSchema):
55. role = Related()
56.
57. class Meta:
58. model = User
59.
60.
61. result = UserSchema().dump(user)
62. print(result)
63.
64. result = UserSchema().load(result)
65. assert isinstance(result, User)
66. assert isinstance(result.role, Role)
67.
I happend to notice that the results 'id' is a string and not an integer unlike in your example code (quickstart).
Schema with custom full_path and url
class FileSchema(ModelSchema):
storage = Related(required=True)
mime = Related()
user_id = fields.Int()
user = Related(load_only=True)
full_path = fields.FormattedString('{path}{name}')
url = fields.FormattedString(S3_ENDPOINT_URL +'/{storage.bucket}/{path}{name}')
class Meta:
model = File
but, when load object
fileSchema = FileSchema()
file, errors = fileSchema.load(payload['file'])
print(json.dumps(file.__dict__, indent=4, sort_keys=True, default=str))
i don't have file_path, only base File model fields
{
"__data__": {
"created_at": "2018-06-28 16:37:19",
"hash": "Y9cR-xxxxxxxxx",
"name": "vid20180619103313.mp4",
"path": "xxxxxxxxxxxxx/",
"size": null,
"storage": null,
"updated_at": "2018-06-28 16:37:19",
"user": xxxxxx
},
"__rel__": {
"storage": "None"
},
"_dirty": "{'user', 'created_at', 'path', 'name', 'storage', 'hash', 'size', 'updated_at'}",
"url": "https://xxxxxxxxxxxxxx/vid20180619103313.mp4"
}
if dump
result, errors = fileSchema.dump(file)
print(type(result))
print(json.dumps(result, indent=4, sort_keys=True))
all ok
{
"created_at": "2018-06-28 16:37:19",
"hash": "Y9cR-xxxxxxxxxxxx",
"id": 7200,
"mime_id": null,
"name": "vid20180619103313.mp4",
"path": "xxxxxxxxxxxxxxxx/",
"size": null,
"storage": {
"bucket": "files",
"id": 13,
"type": "s3",
"url": "xxxxxxxxxx/files"
},
"storage_id": 13,
"updated_at": "2018-06-28 16:37:19",
"url": "https://xxxxxxxxxxxxxxxxxxx/vid20180619103313.mp4",
"user_id": xxxxxx
}
why not in object?
What the point of using marshmallow String
type for peewee AutoField
instead of Integer
?
Is it bug or feature?
Hi!
Sometimes we've got an empty field value in incoming data. And if this field maps on database model field which has default value or null param is set to True, empty value is not good for us. Not always, but very often. I use this code in my ModelSchema wrapper:
class ModelSchema(_ModelSchema):
@pre_load
def drop_empty_fields(self, item):
field_names = tuple(item.keys())
for k in field_names:
field = self.fields.get(k)
if field and not field.required and not item[k]:
item.pop(k)
For example: field is DateField(default=datetime.date.today)
, incoming data value is ""
(empty string). Would be nice to drop this field from incoming dataset and save the instance with default value. Now we get an error Not a valid date
.
I suggest to add a some boolean initial param (for ex. filter
) for manage this case.
What do you think?
I have more that one model schema, each has a Related()
field, which link to another, and so on. Once the object is dumped via a serializer, it only cascades to one related model via the Foreign key, but not to the next model.
Here is an example of the code to replicate the issue:
import peewee as pw
from marshmallow_peewee import ModelSchema, Related
db = pw.SqliteDatabase('example.db')
# Define models
class BaseModel(pw.Model):
class Meta:
database = db
class Planet(BaseModel):
name = pw.CharField()
class Country(BaseModel):
planet = pw.ForeignKeyField(Planet, backref='countries')
name = pw.CharField()
class City(BaseModel):
country = pw.ForeignKeyField(Country, backref='cities')
name = pw.CharField()
db.create_tables([Planet, Country, City])
# Populate some data
planet = Planet(name='Earth')
planet.save()
country = Country(name='Wonderland', planet=planet)
country.save()
city = City(name='Springfield', country=country)
city.save()
# Create serializers
class PlanetSchema(ModelSchema):
class Meta:
model = Planet
countries = Related()
class CountrySchema(ModelSchema):
class Meta:
model = Country
cities = Related()
class CitySchema(ModelSchema):
class Meta:
model = City
# Now dump data using serializer
planet = Planet.get(1)
print(PlanetSchema().dump(planet)[0])
The output from the code above is:
{
'id': '1',
'name': 'Earth',
'countries': [
{
'id': '1',
'name': 'Wonderland',
'planet': '1'
}
]
}
However, the desired output is this one:
{
'id': '1',
'name': 'Earth',
'countries': [
{
'id': '1',
'name': 'Wonderland',
'planet': '1',
'cities': [
{'id': '1', 'country': '1', 'name': 'Springfield'}
]
}
]
}
As you may see, the desired outcome begins from Planet
, cascades to Country
, which later cascades to City
.
If the CountrySchema
serializer is used to dump it, it then includes the CitySchema
as well, but not when the PlanetSchema
is used.
I'm getting this error when trying to serialize queryset with schema that doesn't define fields:
File "...\site-packages\marshmallow_peewee\fields.py", line 79, in get_value
value = obj._data.get(attr)
AttributeError: 'Country' object has no attribute '_data'
and was able to fix it by changing it to:
value = obj.data.get(attr)
OS: win7 pro
Python 3.6.3
peewee:3.2.5
marshmallow: 2.15.0
Marshmallow-Peewee: 1.2.7
Looks like marshmallow changed out from under marshmallow-peewee:
ImportError while loading conftest 'tests/conftest.py'.
[...]
.venv/lib/python3.7/site-packages/marshmallow_peewee/__init__.py:9: in <module>
from .schema import ModelSchema # noqa
.venv/lib/python3.7/site-packages/marshmallow_peewee/schema.py:5: in <module>
from .convert import ModelConverter
.venv/lib/python3.7/site-packages/marshmallow_peewee/convert.py:7: in <module>
from .fields import ForeignKey
.venv/lib/python3.7/site-packages/marshmallow_peewee/fields.py:5: in <module>
from marshmallow.compat import PY2, string_types
E ImportError: cannot import name 'string_types' from 'marshmallow.compat' (.venv/lib/python3.7/site-packages/marshmallow/compat.py)
marshmallow-code/marshmallow@d9bcc48 is the change.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.