Comments (16)
As discussed in the above thread, defaults for deserialization can be defined in make_object
or in a preprocessor
. For example, if you want the deserialization defaults to be the same as serialization defaults, you could do the following:
from marshmallow import fields, Schema
from marshmallow.validate import Range
class GETSchema(Schema):
page = fields.Integer(default=1)
per_page = fields.Integer(default=10, validate=Range(min=10, max=50))
order_by = fields.Select(['id', 'name', 'priority'], default='priority')
sort = fields.Select(['asc', 'desc'], default='desc')
def make_object(schema, in_data):
for name, field in schema.fields.items():
if name not in in_data:
in_data[name] = field.default
return in_data
schema = GETSchema()
schema.load({}).data # {'order_by': 'priority', 'per_page': 10, 'sort': 'asc', 'page': 1}
In lieu of the missing
parameter discussed in previous comments, you can take advantage of the fact that extra kwargs
passed to fields are stored in each field's metadata
attribute. This would allow you to have different defaults between serialization/deserialization.
from marshmallow import fields, Schema
from marshmallow.validate import Range
class GETSchema(Schema):
page = fields.Integer(default=1, missing='null')
per_page = fields.Integer(default=10, missing='null', validate=Range(min=10, max=50))
order_by = fields.Select(['id', 'name', 'priority'], default='priority', missing='id')
sort = fields.Select(['asc', 'desc'], default='desc', missing='asc')
def make_object(schema, in_data):
for name, field in schema.fields.items():
if name not in in_data and field.metadata.get('missing'):
in_data[name] = field.metadata['missing']
return in_data
schema = GETSchema()
schema.load({}).data # {'order_by': 'id', 'per_page': 'null', 'sort': 'asc', 'page': 'null'}
Yes, I am aware that these workarounds are just that: hacky workarounds. I will open up an issue to reopen discussion of a built-in missing
param.
EDIT: Fix first example
from marshmallow.
Yes, I am considering adding the required
check to Schema.load
. Validation upon serialization made sense when marshmallow only had the ability to serialize (but not deserialize) objects. With the new API in 1.0, validation can occur on input data passed to load
. Thank you for the suggestion.
from marshmallow.
OK, so I've implemented the following behavior (now on dev
).
Schema#load
will raise/store an error if a required field is missing from the input dictionary.
class UserSchema(Schema):
name = fields.String(required=True)
email = fields.Email()
in_data = {'email': '[email protected]'} # name is missing
data, errors = UserSchema().load(in_data)
errors # {'name': 'Missing data for required field.'}
data # OrderedDict([('email', '[email protected]')])
Note that None
is still a valid value that may be passed without failing validation.
Schema#dump
will raise/store an error if a required attribute isNone
on the input object.
u = User(name=None, email='[email protected]')
data, errors = UserSchema().dump(u)
errors # {'name': 'Missing data for required field.'}
data # OrderedDict([('name', None), ('email', '[email protected]')])
Schema#load
will use the value of a field'sdefault
attribute if an input value is missing and the field is not required.Schema#dump
will use the value of a field'sdefault
attribute if the input object's corresponding attribute isNone
and the field is not required.
I'm not completely sure whether the dump
behavior is optimal. Some questions left to answer:
- Should
dump
perform the same validation steps thatload
does, including checking required fields? - Should the
default
for deserialization be the same as the default serialization, or should there be anotherField
parameter to allow for two different defaults?
Tagging this one for discussion.
from marshmallow.
Renamed the issue to broaden the scope of the discussion.
from marshmallow.
In case of dump
, when the fields are already according to the internal model, validation is not required at all.
Two separated default
is not exactly needed. I am more inclined to think that the default
values ββare only needed in dump
and not in load
. Otherwise it increases the complexity of code.
from marshmallow.
Thanks @zerodivisi0n for your input.
What if you don't need to use load
but still want to do validation? Take the example of a REST API that allows requesting all objects for a specific resource.
blogs = Blogs.query.all() # no need to use `load`
data, errors = BlogSchema().dump(blogs, many=True)
You raise a valid point though, that validation should occur before serialization and therefore is probably outside the scope of Schema#dump
's responsibility.
from marshmallow.
I've modified the Field
class to only apply user-defined validators on deserialization. The default
parameter only applies to serialization.
I think there may be a use case for a second default parameter for missing input values, similar to the missing
parameter in colander: http://colander.readthedocs.org/en/latest/null.html.
However, if support for preprocessing functions is added (see #47), there would be no need for the new parameter, since missing values could be inserted by a preprocessor.
from marshmallow.
I hesitate to answer what solution is better. It seems to me that additional missing
parameter is over-engineering. But setting default value for single field in preprocessor is bad too. In the case when there is a lot of such fields it requires repeated code for each field. So I guess that missing
parameter is better for these cases.
from marshmallow.
Support for preprocessing functions has been added as of 7f23184. Missing values can be handled by either a pre-processing function or in make_object
, so I won't add the extra parameter for now.
Closing this issue since the proper behavior has been implemented.
from marshmallow.
@zerodivisi0n I didn't see your comment before I posted my most recent comment and closed the issue. Reopening for further discussion.
from marshmallow.
I'm going to hold off on adding the missing
parameter for now, to avoid the possibility of adding unneeded code. I believe a common use case for deserialization will be to use make_object
to construct an app-level object. Defaults can be defined with dict.get
.
class BlogSerializer(Schema):
# fields...
def make_object(self, in_data):
return Blog.query.find_one(_id=in_data.get('_id', None),
title=in_data.get('content, '')
This isn't to say that the parameter won't be added in future releases. A compelling use case may arise that cannot be cleanly addressed using the existing hooks. Until we identify that use case, it will not be included.
from marshmallow.
It's ok to use make_object
for that purposes.
from marshmallow.
But please consider this simple code:
class GETSchema(Schema):
page = fields.Integer(default=1)
per_page = fields.Integer(default=10, validate=Range(min=10, max=50))
order_by = fields.Select(['id', 'name', 'priority'], default='priority')
sort = fields.Select(['asc', 'desc'], default='desc')
How do i set all default values without extra code and much complexity for deserializing(load)?
from marshmallow.
In case of dump, when the fields are already according to the internal model, validation is not required at all.
I get data from a number of sources, mix it with some of my own data, validate and persist. Later, I retrieve the data to do processing. I need to make sure the data's in the expected format, ideally before it leaves this edge system.
Validation would act as a debugging aid, a filter (for bad data), and would notify us if any of the sources changed their data formats before it clogs up the system.
I'd also like to point out that http://marshmallow.readthedocs.org/en/latest/quickstart.html#validation states: Schema.dump() also validates the format of its fields and returns a dictionary of errors.
which I took to indicate that validation does occur when dumping data.
I realize it's probably not easy to re-add validation now that it's been removed for some time, but I'd really appreciate it if this feature were to come back.
If I may present an alternative implementation, having validation as an optional separate step that always operates on Python-native data (before dumping or after loading) would solve my issues.
from marshmallow.
@andrewbaxter Could you use Schema.validate
before serializing?
from marshmallow.
Yeah, thanks, I'm afraid I hadn't noticed that. #189 still is a problem, but I think that's not by design so I'll go back to that issue. I'll open another issue regarding the docs.
from marshmallow.
Related Issues (20)
- Is this safer than Pickle? HOT 1
- Wrong value in __call__() function of validate.Validator HOT 1
- Validation of required nested field is not consistent with previous version
- Create raw list Schema HOT 1
- The function decorated by the "post_load" decorator will undergo changes in parameter types as the function name changes. HOT 1
- Deprecate __version__ and related attributes
- Address deprecation warnings in tests HOT 1
- URL validation fails if URL is missing user field HOT 5
- fields.URL validation error should specify "not a valid FQDN" if trying to pass hostname without domain
- missing `f` for f-Strings HOT 2
- `fields.Url` does not accept `file` URLs
- when a new request, schema context not reset !!! HOT 2
- Questions about Copilot + Open Source Software Hierarchy HOT 3
- Allow None for a allow_none=True Date field HOT 6
- fields.Nested Type Warning Since Version 3.15.0
- Setting many=True through Meta - SchemaOpts HOT 1
- RFE: use `zoneinfo` instead of `pytz` HOT 3
- 3.21.2: pytest fails in units checking time HOT 1
- 3.21.2: sphinx warnings `reference target not found` HOT 2
- Static typing of Field subclass kwargs
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from marshmallow.