Changes to File Overwritten if Changing Schema about ponzu

[nilslice]

First of all, I saw this in Golang Weekly the other day and this is just fantastic. Thanks for making such a cool tool.

Thank you -- I'm happy other people are finding it useful! I appreciate the feedback and hope you get a lot of use from Ponzu.

After playing around with this for a bit, I've noticed that I'm having to edit the generated code quite a bit to customize the admin interface. i.e. I added a profile photo field, so I have to go into the generated code and change the type from editor.Input to editor.File. And then I added a relationship status field, so I had to go in and change it to editor.Select and add my choices.

Yes, this is very typical and although I, too, would like to work more in the CLI, it gets pretty tricky trying to add/insert/remove only parts of a generated file by regenerating a new one. Theoretically it could be done with Go's go/types package which would help navigate your files and the methods, etc you have added. But generally speaking, getting boilerplate out of the way via code generation is enough to satisfy most needs.

I am putting some thought into how the CLI could more easily support an additional specifier to state the field input type.. something like:

$ ponzu gen c profile name:string bio:string:richtext age:int country:string:select ...

The problem is that the developer still has to manually add the select options, etc. I just don't know if the additional complexity in the CLI makes it that much better in the generate files. Also, it would only be able to work with internal element types. See the reference package pre-bundled into your addons directory.. It would be nearly impossible to decipher editor.Select and reference.Select without also including the package identifier in the CLI command.

After making these changes, I then realized I wanted to add another field. When I went to rerun the command, it required me to overwrite my existing changes to generate a new one. Maybe I missed something in the documentation, but there doesn't seem to be an ideal way to handle this scenario.

You should have seen a guard/warn message before anything was overwriten, correct? Something like "please remove 'content/yourcontent.go' before running this command? If not, I'm extremely sorry for any work you lost -- file a bug with your environment, Go version, and as much information about the overwritten file(s) as possible. Again, very sorry if you had code deleted without warning!

Your described usage fits the intended purpose of Ponzu. The generators, much like generators in other frameworks or platforms, write the boilerplate code for you once and you add to it. Changing functions or adding a new field should be pretty painless (although compared to running $ ponzu gen c mycode ..., everything else is kind of painful!). I would recommend copy/pasting as much of the generated code as you can, such as a Field{} or struct field. If you have ideas on how to make this better, at the implementation level, please feel free to help out.

One suggestion might be a way to add, remove, and possibly modify (i.e. type from string to int) fields with the CLI - a schema migration ability of sorts. In the case of removing a field, I may want the data purged from Bolt as well.

There's only so much code Ponzu can write for its users :) I agree that data may need to be purged in the event that the schema changes significantly, and it would likely require a new CLI command, such as $ ponzu migrate which would loop through all the content types and determine which fields stay and go in all Bolt buckets for that data. I'm concerned that this could end up doing more bad than good if used improperly or accidentally, and if you really need a migration like this, you could probably write the code specific to your dataset in a very reasonable amount of time.

On a related note, I feel that, at least optionally, uploading over / removing a file in the interface should delete it from disk. Wth my profile photo example, if someone uploads a new photo, I'd like the old one to be deleted since I've only specified one field for that file.

I believe that the common behavior for most CMS platforms is to keep uploaded content even when it's parent content is deleted. Please correct me if I am mistaken. Many other CMS platforms additionally implement some form of "Media" section where a directory tree of uploads is displayed, allowing uploaded content to be deleted. I'm considering this too - but have not yet had the time or personal need for it. Contributions are very welcome!

Another suggestion might just be to make the tool more robust so that if I want my field to manifest as a select element or file upload element, I can specify that from the CLI. The commands may get long, though, so optional input from a file in a format like JSON, TOML, etc. may be preferable.

100% with you, but as you said, the commands would start to be very long. At what point are we writing just as much code in the CLI as we are in files? The CLI generate content command was deliberately designed to maximize code generated per CLI keystroke, and minimize the potential for mistakes (which would then need to be manually corrected in files).

The input file is an interesting approach -- I'll have to put some thought into that and see how well one of those formats would support this. If you would like to draft a mock config file after having more time to test Ponzu and see where its CLI strenghs and weaknesses are, I'd love check it out.

[ryanbaer]

Wow, GitHub is a better place because of people like you.

Thanks for the quick and incredibly thorough response - really means a lot.

I am putting some thought into how the CLI could more easily support an additional specifier to state the field input type.. something like:
$ ponzu gen c profile name:string bio:string:richtext age:int country:string:select ...

I like that idea and I also see your point about how the select poses an issue. Still, it would considerably reduce the overhead of changing several fields after generating the code if you could specify view details before generating.

It would be nearly impossible to decipher editor.Select and reference.Select without also including the package identifier in the CLI command.

That's good future proofing. Because you thought to include addons (which is a great idea), you would have to namespace which could get fairly verbose. I suppose you could also make them globally unique (i.e. ReferenceSelect) since you have to register the add-on anyway, but I haven't given too much thought to the ramifications of that.

You should have seen a guard/warn message before anything was overwriten, correct?

Sorry, didn't mean to alarm you - when I said "required me to", I meant that it prompted me in the way you just described. I didn't lose any data :)

I'm concerned that this could end up doing more bad than good if used improperly or accidentally, and if you really need a migration like this, you could probably write the code specific to your dataset in a very reasonable amount of time.

That's true - things could get messy and complicated pretty quickly if you introduced this. And it feels a bit out of scope now that we've discussed it.

Many other CMS platforms additionally implement some form of "Media" section where a directory tree of uploads is displayed, allowing uploaded content to be deleted.

That does sound familiar and what you're describing sounds like a reasonable solution. It was more that I realized they could pile up over time and didn't see a way to manage that in the interface, but the media section would resolve that.

The input file is an interesting approach -- I'll have to put some thought into that and see how well one of those formats would support this. If you would like to draft a mock config file after having more time to test Ponzu and see where its CLI strenghs and weaknesses are, I'd love check it out.

Hmm OK I'll include some examples for discussion at the end of this reply.

---

Thanks again for addressing all of my points. As far as this issue is concerned, I think an input file could help resolve it if it allowed the user to specify the view type as well. Since you requested it, I came up with quick examples in TOML and JSON based on 2 different possible approaches. Let me know what you think.

Both examples address what it could look like to specify the view and even view choices in the case of a Select element.

Example 1: String-based Approach
This approach is essentially just a newline-friendly version of the CLI. In implementation, you would still have to split on colons and optionally commas in the case of a Select.

• JSON Example
• TOML Example

Example 2: Structure-based Approach
This approach utilizes the structure we have access to in JSON and TOML. Rather than splitting on commas and colons, each feature has its own field. In particular, the separate array for viewChoices would allow someone to more easily specify and manage a larger number of choices (all US states, for example).

• JSON Example
• TOML Example

[nilslice]

@ryanbaer -

Sorry for the delayed response - I have a couple major deadlines coming up quickly.

I think adding the optional specifier to the CLI items is a good idea and will plan to add it in a future release - however, it will likely be limited to just the builtin editor package input types - hopefully this will accomplish the goal of making the CLI more useful without adding bloat.

Although the idea of using an input file as a schema definition for the types is appealing, I don't think that it is really much less typing that manually writing out code. The benefit I do see in using something like String-based approach is that we could easily share the input schemas between projects and developers as a standard... I'll have to put more thought into this, but at the moment I think we'll just keep it simple -- maybe once the project matures and more people make similar suggestions we will have clarity on how this kind of feature should be implemented.

Also, so you are aware, I am starting to migrate feature suggestions to Github's Project feature to clean up issues: https://github.com/ponzu-cms/ponzu/projects/2

Thanks again for your thoughtful feedback and I look forward to working together on some ideas for Ponzu!

[nilslice]

@ryanbaer - just wanted to ping you to say the enhanced CLI code generator that you suggested has been added and is the latest master branch of the Ponzu repo. Sorry it took so long ;)

lmk if you get a chance to try it out. currently, only the basic inputs are supported (excluding repeaters, since I'm not 100% sure what the right nomenclature would be.) Check out the docs for these "Input View Specifiers" in the README here.