Search endpoint - request format about plone.restapi

[tisto]

@lukasgraf thanks for the writeup! I tried to wrap my head around this issue as well and haven't reached a final conclusion yet.

You can ignore searching.rst. This was a very early draft before we even looked at existing solutions. I will try to find some time to do some more research. Though, this seems to be a topic that most API architects seem to avoid talking about. ;)

Just a remark regarding POST vs. GET. It seems that some people argue that you actually create a search with a query and therefore a POST request can also be correct in REST terms. Though, you are of course correct with the other disadvantages (caching, no URL, etc.).

[bloodbare]

Great job @lukasgraf !

I love your ideas, in some projects we are using JWT (jwt.io) to have the search query on a parameter as they can be complex. If we wanna support complex queries , the parameters get complex and jwt allows to define a JSON format.

I would support only GET verb.

It may be an option to have to endpoints ? one for text search and one for complex search ?

[lukasgraf]

@tisto

Just a remark regarding POST vs. GET. It seems that some people argue that you actually create a search with a query and therefore a POST request can also be correct in REST terms. Though, you are of course correct with the other disadvantages (caching, no URL, etc.).

Yes, I've also seen POST being used. I created a separate issue #46 for discussing a potential concept to maybe address some race conditions with batching - that would require a POST anyway.

@bloodbare

If we wanna support complex queries , the parameters get complex and jwt allows to define a JSON format.

Yes, for any non-trivial query I don't really see anything besides JSON as an option any more. We really don't want to write our own query DSL.

I would support only GET verb.

We (4tw) also have a strong tendency towards that, if we can make it work.

I'd argue that then means that 3) (GET + JSON in query string) emerges as a promising option.

It may be an option to have to endpoints ? one for text search and one for complex search ?

Yes, we also considered that. For now I'd like to focus on an implementation that can gracefully handle complex queries, because that's the hard case, but I could very well see a limited simple_search endpoint with a simplified syntax and reduced functionality.

[jone]

I'd love to have a simple JSON-payload rather than having to build a compliacated GET-querystring.
Because GET + payload is a problem I'm therefore in favor of using payloads with POST-requests (nr. 4.)

[lukasgraf]

For 3) it would be about as complicated as this 😜

query_json = quote_plus(json.dumps(query))
requests.get(url, params={'q': json_query})

Or with curl:

 curl -G -H 'Accept: application/json' $URL --data-urlencode 'q={"SearchableText": "foobar"}'

Or with httpie:

http -j GET $URL q=='{"SearchableText": "foobar"}'

Maybe offering both is the way to go? As long as the serialization format is JSON for both, implementation and documentation overhead should be minimal.

[jone]

That's for me ok as well. I think we should just implement a first version; it can be extended later..

[lukasgraf]

Updated:
The use of JSON only allows to retain some typing information, but not all (datetime isn't JSON serializable).

This significantly diminishes the advantage of the use of JSON as the query serialization format, because we still need to have a mechanism in place to reconstruct typing information based on knowledge about the index that's being queried.

Under these circumstances I really don't see a point in also offering POST anymore. While on the implementation side it's trivial to also support POST, it still needs to be tested and documented. Also, offering two ways to do the same thing goes against the Zen of Python There should be one-- and preferably only one --obvious way to do it., and might confuse our users.

For this reason, I think GET with query string and optional Zope type hints is the way to go. I'll update my search PR (#48) accordingly in the next few days and propose a GET only implementation.

[lukasgraf]

A first version implementing GET with query string has been merged with #48