plone / plone.restapi Goto Github PK
View Code? Open in Web Editor NEWRESTful API for Plone.
Home Page: http://plonerestapi.readthedocs.org/
RESTful API for Plone.
Home Page: http://plonerestapi.readthedocs.org/
{
@type:
total_items: 42,
items: [],
batch_actions: [
{
'name': 'previous',
'title': 'Previous',
url: 'http://...'
},
{
'name': 'next',
'title': 'Next',
url: 'http://...'
},
{
'name': 'first',
'title': 'First',
url: 'http://...'
},
{
'name': 'last',
'title': 'Last',
url: 'http://...'
}
]
{
contentType:
size (kb)
data/download
filename
}
It should be possible to configure custom media types in addition to using 'application/json'.
Not sure if this makes sense without a custom serializer.
The schema serialization is currently quite hard-coded for the standard fields.
When having custom fields, it is only possible to provide serialization by copying the serialization function and adapting it.
I'd like to change the serialization mechanism so that I'm able to change how a certain field value is represented by registering an adapter and easily provide custom serializer for my custom fields.
Maybe something like that:
class IFieldSerializer(Interface):
"""The field serializer multi adapter serializes the field value into
a JSON compatible python data. """
def __init__(context, request, field):
"""Adapts context, request and the field.
"""
def serialize():
"""Returns JSON compatible python data.
"""
Although the request is not really relevant here, I usually like multi adapters to also adapt the request so that I'm able to easily register a more specific adapter by using a (project-) request-layer as discriminator. Without adapting the request I'm more often forced to customize with an overrides.zcml
, which I do not like. This is my personal taste and experience though ๐
Some questions:
plone.restapi
or into another package?After short discussion on #plone i want to raise this up as an issue here. Currently my post-requests to backend fail after upgrading to plone.protect >= 3.0
because i do not ship a token within the request.
How do you guys handle this?
Create a minimal JS app that makes the API explorable. First this could be a read-only browser. At a later point we could use actions and the schema to auto-generate forms (like Django REST Framework).
I get:
Application Error
An error occurred in the application and your page could not be served. Please try again in a few moments.
If you are the application owner, check your logs for details.
when trying to access: http://arcane-sierra-8467.herokuapp.com/Plone/@@json
There are different ways to model the parent/child relationship for our JSON objects. Possible attribute names:
__parent__
/ __children__
(pro: parent is already used in Zope/Plone, no name collisions, con: ugly?)If I do an http get on a collection (which I have called "hey") with two items. It returns information about the two items in the "member" array. However for each member I only get somethign like this:
"member": [
{
"@id": "https://mysite.example.com/hello",
"description": "is it me you're looking for (upspeak)",
"title": "hello"
},
{
"@id": "https://mysite.example.com/front-page",
"description": "Congratulations! You have successfully installed Plone.",
"title": "Welcome to Plone"
}
],
I would want the content type added as @type to the metadata.
"member": [
{
"@id": "https://mysite.example.com/hello",
"@type":"News Item",
"description": "is it me you're looking for (upspeak)",
"title": "hello"
},
{
"@id": "https://mysite.example.com/front-page",
"@type":"Document",
"description": "Congratulations! You have successfully installed Plone.",
"title": "Welcome to Plone"
}
],
For completeness I've included the full JSON output below
{
"@context": "http://www.w3.org/ns/hydra/context.jsonld",
"@id": "https://mysite.example.com/hey",
"@type": "Collection",
"UID": "c650b8b9a05343abaa65cade44150c1f",
"allow_discussion": null,
"contributors": [],
"created": "2016-04-17T14:23:01+00:00",
"creators": [
"manager"
],
"customViewFields": [
"Title",
"Creator",
"Type",
"ModificationDate"
],
"description": "",
"effective": "2016-04-17T14:23:00",
"exclude_from_nav": false,
"expires": null,
"id": "hey",
"item_count": 30,
"language": "en-gb",
"limit": 1000,
"member": [
{
"@id": "https://mysite.example.com/hello",
"description": "is it me you're looking for (upspeak)",
"title": "hello"
},
{
"@id": "https://mysite.example.com/front-page",
"description": "Congratulations! You have successfully installed Plone.",
"title": "Welcome to Plone"
}
],
"modified": "2016-04-17T14:51:22+00:00",
"parent": {
"@id": "https://mysite.example.com",
"description": "",
"title": "Site"
},
"query": [
{
"i": "portal_type",
"o": "plone.app.querystring.operation.selection.any",
"v": [
"Document"
]
}
],
"relatedItems": [],
"rights": null,
"sort_on": null,
"sort_reversed": false,
"subjects": [],
"text": {
"content-type": "text/html",
"data": "<p>here<strong> is</strong> some <em>tect</em> for you</p>",
"encoding": "utf-8"
},
"title": "hey"
}
Make links "discoverable" for a possible client. This will allow a loose coupling between client and server (e.g. we can change the URLs without breaking the client).
A JSON-LD schema allows us to define which parts of the JSON document are actually links. A client can then expand the JSON-LD document and follow links (with something like has_link, follow_link, take_action).
As discussed at the Barcelona Sprint, there's going to be a first alpha release of plone.restapi
soon.
These are the current blockers that are left:
Deferred for now:
GET
response differently, with fields in their own nested dictionary (issue #6, needs discussion)For reference, this was the state of the Kanban board was in when we left the sprint yesterday:
@tisto @buchi @sneridagh did I forget anything?
@tisto told me you @bloodbare were working on some CSRF issues, i assume it was in ramon branch - can we use as a general guideline to use more expressive naming convention for branches.
I'm currently changing the serialization as discussed in #38.
While looking at the current implementation I was wondering why iterating over dexterity fields is implemented that way.
See get_object_schema
:
def get_object_schema(obj):
# Iterate over all interfaces that are provided by the object and filter
# out all attributes that start with '_' or 'manage'.
for iface in providedBy(obj).flattened():
for name, field in getFields(iface).items():
no_underscore_method = not name.startswith('_')
no_manage_method = not name.startswith('manage')
if no_underscore_method and no_manage_method:
yield name, field
# Iterate over all behaviors that are assigned to the object.
assignable = IBehaviorAssignable(obj, None)
if assignable:
for behavior in assignable.enumerateBehaviors():
for name, field in getFields(behavior.interface).items():
yield name, field
I would have used plone.app.dexterity.utils.iterSchemata
with zope.schema.getFieldsInOrder
(or zope.schema.getFields
).
@tisto what is the advantage of this approach compared to using the dexterity utils iterSchemata
?
Right now only PATCH
is implemented, which takes a diff of the object's representation as input for what fields should updated.
We also want to implement PUT
, which would take the objects complete representation.
Discuss: What should happen if some fields are omitted by the client?
Instead of iterating over all possible interfaces, just iterate over the dexterity type schema and all behaviors.
The import of ICollection
in the serializer code introduced in #39 adds a new, implicit dependency on plone.app.contenttypes
.
Until now, plone.app.contenttypes
is only a test dependency, and I'd really like to keep it that way.
Can we use an alternative to the ICollection
interface to recognize collections?
If you have a frontend app for example running on port 9000 and it requests the plone backend on localhost:8080 you will get:
XMLHttpRequest cannot load http://localhost:8080/plone/@@json. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:9000' is therefore not allowed access.
This is CORS issue. The frontend address has to be registered as an allowed ressouce in the plone backend (zope server).
IMHO a CORS TTW configuration is desirable. Perhaps it is a good idea to have an isolated package plone.cors which offers a configlet and a server response patch.
In the Pyramid world The Cornice REST framework has to deal with this issue and can simply be configured: http://cornice.readthedocs.org/en/latest/api.html
In The Django world a so called middle ware can be registered and configured in settings.py: https://github.com/ottoyiu/django-cors-headers
Background:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
http://www.html5rocks.com/en/tutorials/cors/
http://enable-cors.org/
The JSON Schema produced by @types/[portal_type]
currently only includes the field type, but no information about which widget should be used. This information should be included in the JSON Schema as well.
A custom widget for a field can be specified using a plone.autoform
directive. That information then gets stored on the schema interface using tagged values, which should be easy to access and serialize.
The default widget however for a particular zope.schema
field type is currently being determined by z3c.form
, which means we might need to replicate some of z3c.form
's logic.
/cc @ebrehault
The repository plone.restapi has no license.
Are there any existing plone.* package that can provide content negotiation? This is needed to provide web-services with the constraints set forth by HATEOAS (which I noted that it is finally included as a consideration, which satisfied some of my original concerns about Plone's approach of web-services).
One way to get around this issue specific to Plone in one of my projects is to have a subscriber to the IBeforeTraverseEvent
event for the site root. The subscriber will then dynamically mark the request with layer(s) from matching rule(s) (i.e. the mime-type at hand) as set forth by any of the registered layer utilities. These layer utilities can then be provided by the implementation packages for any of the JSON web-services oriented content types to register their availability within Plone.
Anyway, this issue is mostly a question on what to do, as I noticed that the design documentation haven't really addressed this point yet or any possible approaches to solve this problem, or whether this is already the approach already being considered on some other discussion lists elsewhere.
Discussion notes from REST API sprint in Barcelona. (@vangheem @ebrehault @tisto)
JSON Schema currently supports:
In addition we need:
First implementation:
If the client sends a GET request to the "@types" endpoint with the name of the content type and an HTTP header "Accept: application/schema+json" the server will respond with a JSON schema document.
I think the api should check if attributes are accessible (i.e. check for field-permissions plone.autoform.directives.read_permission
)
We want to integrators to be able to build nested resources (e.g. '/university/1/faculty/3'). In addition we might want to model certain Plone specific resources in a nested fashion as well (e.g. '/document1/comment/23').
Just to keep somewhere what we discussed at Barcelona Sprint with @tisto and @vangheem:
The append_json_to_links
utils function seems not to be used at all.
Can we remove it?
{
contentType
size
data/download
filename
scale: {
'mini': {
href: ...
width: 400,
height: 200,
}
}
}
There have been some changes to plone.app.contenttypes fixture that make it hard to upgrade.
/cc @ebrehault
{
@type: PortalRoot,
global_actions: [
'name': 'search',
'title': 'search',
'href': 'http://nohost/search',
'schema': our own search schema (schema.org)
]
}
(I'm just dumping this here to not have the conversation in #45 get too convoluted - for now I see this as low to medium priority).
Once we implement some sort of batching / pagination, there's some inherent race conditions that can occur:
Imagine a search query. Because fetching a batch page happens in a separate request, the extent and order of the resultset for a given query can change between retrieving batch pages if another client modified the DB in between. This can lead to either duplicate entries or entries that got dropped between batch pages when a consumer simply iterates over all entries in all batch pages.
ElasticSearch addresses this in a rather elegant way with its Scroll API:
_scroll_id
that uniquely identifies the resultset created by the query at that point in time_scroll_id
. On each of those requests the TTL for the search context is reset, so it is kept alive for another $TTL
minutes.I could see a similar concept working for us in order to provide stable resultsets for batched sequences, particularly search results.
I'm just brainstorming here, but maybe something along these lines could work:
POST /Plone/search
{"portal_type": "Document"}
This would create a server side, persistent search context. In terms of search results, this could maybe mean persisting a list of brain RIDs [1] for the resultset that matched the query at that point in time.
Returns a response with a scroll_id
:
{"scroll_id": "f40dba5"}
The client then can retrieve result batches via GET
requests:
GET /Plone/search?scroll_id=f40dba5&page=1&per_page=20
The link to the first batch page can also be provided in a hypermedia fashion as part of the response to the POST
that creates the search context.
Search contexts that exceeded their TTL would be destroyed with the next POST
. In addition, they could be actively cleared by the client using DELETE
or PURGE
.
Compared to a simple, stateless GET
implementation, I see these pros/cons:
Advantages:
POST
bodyGET
with query string paramsDisadvantages:
[1] Is there a way to get the brain RIDs from a catalog resultset (LazyMap
) without destroying its lazyness? If not, that would at least partly defeat the purpose of batching ๐ข
Proof-of-concept implementation to make the ZPublisher support the following HTTP verbs:
When the client sends the 'application/json' Accept header and an error occurs, the ZServer should respond with a JSON encoded error message.
We should support relation fields.
I think we should have a general short-form for representing content which can use when representing relations but also for children, parent and more.
How do we do that?
Is there a way to change the workflow state of an object with plone.restapi? In the documentaton (http://plonerestapi.readthedocs.org/en/latest/introduction.html) I didn't see anything about it.
I'd like to start off the discussion about implementing a service that handles searching / querying the Plone site, and hope that we can come to a decision on the fundamental points so I can start with an implementation.
I already took a stab at implementing the way search is currently outlined in docs/source/collections/searching.rst. However, I've encountered a couple issues with this approach. So I evaluated several different possible implementations, each with their own set of issues.
Common pros/cons:
GET
:
POST
:
Note that in all examples that use a query string the parameters would need to be URL escaped.
GET
with query params in query string (no type hints)Example:
search?SearchableText=lorem&path.query=/Plone/my-folder&path.depth=2
This approach is using GET
requests with parameters in query strings, without using Zope style type hints (:record
, :list
, etc.). Parameters that contain a dot in their key will be merged into corresponding dictionaries.
Advantages:
Disadvantages:
Arguments to index-specific query options can't be typed. This is the big one. Imagine a query like ?path.query=/Plone&path.depth=0
. The 0
value for the depth
option needs to be an integer when passed to catalog.searchResults()
, otherwise the ExtendedPathIndex
will fail with a TypeError
. Given the query string syntax without the Zope type hints, there is no way for the API consumer to declare a type for these arguments, they'll all end up as a string in request.form
.
This means that, even ignoring validation for now, with this approach the server side needs to do some type conversions. This is tricky, because there's no programmatic way to determine the required types for these type of index-specific options. All you get is a listing of query_options
.
So we'd need to maintain a list of information objects that describe the required types of query options, not unlike to what @vangheem has done in collective/elasticsearch/indexes.py
. These would need to cover at least the index types present in a standard Plone, and support some kind of extension mechanism (adapter lookup) to deal with other index types.
I've gotten an approach like this to work, but it's not pretty, and I wouldn't be looking forward to maintaining that.
GET
with query params in query string (plus Zope type hints)This would be the exact same style of query string that Plone currently uses for its @@search
view.
Example:
search?SearchableText=lorem&path.query:record=/Plone/my-folder&path.depth:record:int=2
Advantages:
Disadvantages
TypeError
with the value (but no key) is usually all you getGET
with an URL encoded JSON doc as single query string paramExample:
search?q={"path": {"query": "/plone/folder", "depth": 0}}
(Obviously would need to be URL encoded)
For the query, there's a single query string parameter q
that contains an URL encoded JSON document that, when deserialized to a Python dictionary, contains a query that can be handed off to catalog.searchResults()
.
Advantages
Disadvantages
POST
with query as JSON document in bodyExample Request:
POST /Plone/search
{
"path": {
"query": "\/plone\/folder",
"depth": 0
}
}
Advantages:
datetime
)searchResults()
and send it on its way with requests.post(url, json=query)
.Disadvantages:
POST
is the wrong HTTP method. This might
POST
requestsGET
with query as JSON document in bodyI briefly considered this as an approach (with a POST
alternative), but the fact is, HTTP clients can't deal with GET
+ body very well, and ZPublisher
can't either. So I think this is already out of the question, just mentioning it here for completeness.
Advantages:
datetime
)Disadvantages:
ZPublisher
without some trickery@tisto @bloodbare
So, moving forward, do you guys see any other options that I haven't covered? Should we swallow the red pill and continue implementing search as outlined in docs/source/collections/searching.rst, accepting that we need to maintain a list of index descriptions? Or do you see one of the mentioned approaches as a viable alternative?
Also, it would be useful to document which fields are available/required when POSTing.
Create a base class for REST API Exceptions and have existing exceptions raised by the API inherit from it.
Having Plone.restapi (plone.restapi 0.1, plone.rest = 1.0a5) installed breaks the file upload functionality in a stock Plone 5 (at least in versions 5.0.3/4). Dragging files into the upload area and pressing upload yields the text "The file upload failed" just below the filename and nothing happens.
I have installed plone restapi with:
`auto-checkout = plone.restapi
eggs =
Plone
Pillow
plone.restapi
[sources]
plone.restapi = git https://github.com/plone/plone.restapi
`
and running buildout -c develop.cfg.
Disabling the restapi egg restores file uploads.
The Plone error log show two errors (after clearing "Ignored exception types"):
`Traceback (innermost last):
Module ZPublisher.Publish, line 127, in publish
Module ZPublisher.BaseRequest, line 523, in traverse
Module ZPublisher.HTTPResponse, line 727, in debugError
NotFound:
An error was encountered while publishing this resource.
Debugging Notice
Zope has encountered a problem publishing your object.
Cannot locate object at: http://localhost:8080/testing/my-docs/fileUpload
Troubleshooting Suggestions
For more detailed information about the error, please refer to the error log.
If the error persists please contact the site maintainer. Thank you for your patience.
`and:
Traceback (innermost last): Module ZPublisher.Publish, line 127, in publish Module ZPublisher.BaseRequest, line 508, in traverse Module ZPublisher.BaseRequest, line 332, in traverseName Module zope.traversing.namespace, line 112, in namespaceLookup Module Products.CMFPlone.traversal, line 39, in traverse Module plone.resource.traversal, line 27, in traverse NotFound
Currently the script doesn't check if the user is permitted to view the fields on a page (custom permission). Is something like this implemented or on the todo list already?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.