krakend / flatmap Goto Github PK

Current operation types allow you to rename (move) or delete properties, but not to define which are the fields you want to pick.

If the move operation would let you take an object outside the current scope, picking a field would be as easy as { "type": "move", "args": [ "results.*.desired", "*.desired" ]} and { "type": "del", "args": [ "results" ]},. As the move does not support moving properties outside the scope, the semantics of writing "I want this field" are really verbose:

Verbose example

For instance, let's take the Get Popular Movies from The MovieDB API. Say that I want to keep only title and id from the results, then I would need to define a flatmap like this (delete every single field I don't want):

{ "type": "del", "args": [ "results.*.adult" ]},
{ "type": "del", "args": [ "results.*.backdrop_path" ]},
{ "type": "del", "args": [ "results.*.genre_ids" ]},
{ "type": "del", "args": [ "results.*.original_language" ]},
{ "type": "del", "args": [ "results.*.original_title" ]},
{ "type": "del", "args": [ "results.*.overview" ]},
{ "type": "del", "args": [ "results.*.popularity" ]},
{ "type": "del", "args": [ "results.*.poster_path" ]},
{ "type": "del", "args": [ "results.*.release_date" ]},
{ "type": "del", "args": [ "results.*.video" ]},
{ "type": "del", "args": [ "results.*.vote_average" ]},
{ "type": "del", "args": [ "results.*.vote_count" ]},

Desired functionality

The desired operation should allow an easier way to pick the desired fields and delete everything else.

For instance:

{ "type": "whitelist", "args": [ "results.*.id", "results.*.title" ]},

Keeps id and title inside the array and deletes any non-matching element.

The whitelist operation deletes all non-matching elements. Meaning that it makes no sense to stack several whitelist operations.

This is probably too much magic... but worth reviewing.

I was wondering if it would make sense to support copy as a new type of operation.

One of the use cases I am thinking of is when you want to extract nested parameters for a sequential proxy temporarily but you still need the response as is. The move is going to remove the element from its position, so it doesn't cover the case.

Sequential proxy could also support nested parameters, but not sure if I would do it.

Thoughts?

the performance of the current implementation of the Expand method is "sub-optimal" and it decays as the deep in the managed structs and the size of the contained collections increases.

So collections and maps could be appended into a single one

Ex: append collection2 collection

original:

{
	"collection": [1,2,3]
	"collection2": [10,20,30]
}

result:

{
	"collection": [1,2,3,10,20,30]
}

Check out https://github.com/twpayne/flatjson - it seems quite similar to what you're working on here.

Feel free to close this issue as spam :)