Please Support TiDB about ponzu

[krismeister]

Sorry I was thinking the REST interface not admin related.
I don't want to overstep myself, but I do care about this feature, and needed to hack it manually the on a recent headless CMS site.

Simpler Publish/Draft only

The ability for browsing creating or scheduling releases may be easier to split into 2 phases of work. The first step could be to have 2 versions only: the Published version and the Draft version.

Firstly drop the idea that on document save it publishes immediately. Instead when an administrator saves a document, it gets added to the draft version. They can continue to make many edits across models which continue to get added to the Draft version. Finally they can 'publish' all changes in the draft version to the publish version.

The admin UI which may work is something more global. Like a horizontal bar across the admin.

No Draft Edits in progress

+-----------------------------------------------------+
| All Edits are Published                             |
+-----------------------------------------------------+
+-----------------------------------------------------+
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
+----------------+                                    |
                 |                                    |
                 |                                    |
                 |                                    |
                 |                                    |
                 |                                    |
                 +------------------------------------+

A user has made document edits/saves.

+------------------------------------------+----------+
|In Draft                          Details | Publish  |
+-----------------------------------------------------+
+-----------------------------------------------------+
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
|               ||                                    |
+----------------+                                    |
                 |                                    |
                 |                                    |
                 |                                    |
                 |                                    |
                 |                                    |
                 +------------------------------------+

A user views details of the Draft

+------------------------------------------+----------+
|In Draft                           Details | Publish  |
+-----------------------------------------------------+
+-----------------------------------------------------+
|               ||                                    |
| Viewing Draft ||  Draft Details                     |
|               ||                                    |
|               ||  5 Edits:                          |
|               ||  XXXXX                             |
|               ||  XXXXX                             |
|               ||  XXXXX                             |
|               ||  XXXXX                             |
+----------------+  XXXXX                             |
                 |                                    |
                 |  Delete Draft | Publish Draft      |
                 |                                    |
                 |                                    |
                 |                                    |
                 +------------------------------------+

More Complicated / Future Release Functionality

The Publish/Draft only method might be a good first step for ponzu. More complicated functionality would support Full Release Planning.

Below is the way which releases edits could show in the admin UI.

The bar at the top would have a dropdown to select which release you're currently working on, plus additional one actions buttons, view-details, and publish.

Global Dropdown to which release your edits are saved to

+------------------------------------------+----------+
|Release XXXXXXX  V                Details | Publish  |
+-----------------------------------------------------+

Here is a detail view of a release

+------------------------------------------+----------+
|Release XXXXXXX  V                Details | Publish  |
+-----------------------------------------------------+
+-----------------------------------------------------+
|               ||                                    |
| Releases      ||  Release Details XXXXXX            |
|               ||                                    |
| XXXXXX        ||  5 Edits:                          |
| YYYYYY        ||  edit abc                          |
| ZZZZZZ        ||  edit foo                          |
|               ||  edit bar                          |
| Create Rel.   ||  edit cow                          |
+----------------+  edit dog                          |
                 |                                    |
                 |  Delete Release |  Publish Release |
                 |                                    |
                 |                                    |
                 |                                    |
                 +------------------------------------+

[joeblew99]

Just want to add that there is another possibility here to allow other DB's and in fact other microservices to use Ponzu.

If we think of Ponzu as a gateway it allows to use data in other systems. The value of ponzu is that it gives a GUI to types and references types. And now also allows updates from that GUI.

But it would not be a huge jump to allow certain types to be Get and Set from other systems (like micro services) etc etc. This allows devs to store their data in other DBs or microservices, and keep Ponzu acting as a GUI gateway.

DOnt knwo what others feel about this. It does not at all stop devs just sticking with ponzu and its boltdb internally, but allows interacting with other systems that expose types too.

[nilslice]

@krismeister - wow, thank you for putting that together & I apologize for missing it somehow. I will have to spend some time with the layout and trying to fit the whole idea into a feature.

@voxadam - I think the steps to make document revisions happen would be first to get it sketched out in Bolt where we can complete the feature and make sure it integrates nicely. Then, once we are ready to tackle the additional database support we can extend it to work with a graph database. I'm cautious about putting any new database-level dependencies into the project until then.

[nilslice]

Hi @sebastianmacias -

I am not familiar with TiDB, but upon first glance, it would appear that in order to keep an all-Go system + deploy a single binary, BoltDB would still be the underlying storage engine, correct?

There is already some interest from the community in working on support for multiple DB backends. If you have thoughts on how to accomplish this, please add notes to this issue and I will add an official project for the topic soon.

If you have not already, take a look through the system/db package. I am wondering if the API exposed could support a lower-level implementation to swap the underlying DB behind package functions like db.SetContent etc., or if there would need to be a significant overhaul and redesign of the package (which would possibly justify planning this for a 2.0 release)

Thanks!

[nilslice]

Hi @sebastianmacias -

I wanted to see if you had any more thoughts on this. Otherwise, I'd like to close this issue but add a deferred label to it so we can easily bring it back up once there is more energy behind the discussion around database backends.

Thanks,
Steve

[voxadam]

If you're entertaining the idea of changing backends or supporting multiple backends you might want to take a look at Dgraph too.

[nilslice]

@voxadam thanks for the tip -- we're planning to keep Bolt as the primary backend, but it would be nice to restructure things a bit to eventually allow for others. I'd need more help on that matter though, certainly not a job for myself alone.

[sebastianmacias]

I think a pluggable datastore architecture would allow the most flexibility and awesomeness. Being able to chose a golang-based database engine (bolt, TiDB, dgraph, etc) and still have the simplicity of a single binary deployment would make this amazing.

[krismeister]

Does any golang DBs store document revisions?

Otherwise in designing off BOLT as the first implimentation, if the admin an area ever wanted to show document history, Ponzu would have to do the revision ID juggling manually, where as some noSQL DBs store all content revisions and are programmatically easy to retrieve.

[nilslice]

@krismeister - I'm not aware of one in pure Go. Will keep an eye out though and maybe add "revision history" to the Community Feature Suggestions project on Ponzu's github projects.

[faddat]

@krismeister

Noms stores document revisions, but keeps all of the state on all of the machines. I would love to use noms with Ponzu, though. Here is what I've considered as possible distributed back ends to ponzu:

• noms
• cockroachdb (scale issues-- couldn't do 1000s of nodes)
• tidb (has rust dependencies)
• dgraph (is awesome, but wish that it worked a little more like GunDB. Right now it's RAFT. I don't think it could achieve the kind of scale we need but I should look into this more.)

Another project similar to Ponzu is gu-io. The gu projects seem to be @joeblew99 & @influx6's manifesto against native front end development.... sort of. One of the theories is well, if we have go, and tools like go-qt, we can provide actual, decent native apps for * without too much of a tax. I like the thesis as much as I do Ponzu's thesis, yup!

[joeblew99]

GU is not that similar to Ponzu. Totally different architecture. Just saying so there no one burning he candle of hope ....

Ponzu and Revisions and different DB's etc.
if ou want an audit trail or everything someone did use a eventstore pattern, NOT a DB

[krismeister]

Eventstore is that each edit is written into a new table?

The advantage of a DB which versions and merges. Is setups like A-B testing or content releases can be done easier. If a content author wanted to make many edits across content, preview those edits, then publish all the edits in bulk, a DB schema which can version and merge is needed. Some DBs have this natively.

[nilslice]

Due to the mechanics of how Ponzu manages changes to content (see the merging func in system/db/content.go), it would be fairly straitforward to assign a new bucket per content type and stash the previous content data in a sub bucket or something that assigns a increments key per update.

Unsure how to go about designing the front-end display and UI for listing and loading those revisions though.

[krismeister]

On the FE, the way it's often handled, is to make a dedicated url, which sets a session cookie for a version and redirects to another page. From then on during that session the rest GET calls pass a revision identifier either by url param or request header.

[nilslice]

Thanks, @krismeister - do you know how the admin user would select a different version in the UI? Is it a drop down with version number and dates, a list of versions somewhere that link to the revision? Does clicking the link to the version automatically pin it as the "current version", etc.

Lot of behind the scenes stuff to consider (as usual! haha)

[joeblew99]

@krismeister @nilslice

Doing a bucket with versioning is a good way too.
Event stores is what i use because i use CQRS, so its my own bias.

[nilslice]

@joeblew99 - I'll have to take a look at that, thanks for the info!

[voxadam]

It should be possible to maintain document revisions in pretty much in pretty much any graph database but the schema might get a bit hairy.

[olliephillips]

This issue is very old. Going to close. Please feel free to reopen if needs be.