Is there a way to save all the undoInfo when packing? about zodb

[gotcha]

I guess that you know that when you pack, you loose the possibility to undo to any of the states removed by packing.

I'd be interested to know why you want to keep the information.

[idgserpro]

This information is interesting to have to know the actions that users had with different urls, when deleting, publishing and such. In simple terms, an auditing perspective of modifications that were made to the database that's available out of the box when you just use the ZODB.

I don't know if this is out of the scope of ZODB - I'm interested in this feature from a Plone/Zope perspective. That's why I'm asking it here to know your opinion on this matter.

If it's not in the interest of ZODB developers and zopefoundation to have a feature like this, at least we now have a historic record of reasons that show why it shouldn't be done.

Thanks in advance.

[jimfulton]

Undo information comes from the data that gets packed away, so no, there's no built-in way to keep undo information after packing.

[idgserpro]

@jimfulton what is your opinion about this feature? Disadvantages, complications and such? Why it shouldn't be done? Right now there's no built-in way to keep undo information after packing, but we think it would be a useful thing to have it like it's explained in our last comment.

[jimfulton]

I am ambivalent. I'm certainly not going to implement it.

[idgserpro]

@jimfulton I know this is not priority for your development, we're just afraid of trying to sponsor such development and have a rejected pull request if the developers don't think this feature belongs to ZODB. At least a historical record of "why this shouldn't be done" would be really nice.

Who do you think we can add to help in this discussion from https://github.com/zopefoundation/ZODB/graphs/contributors?

[jimfulton]

I don't have any problem with your use case at a high level. It's not clear that it requires a change to ZODB. If you want it badly enough to fund development, I suggest approaching it as an add on that can work without changing ZODB. A good place to discuss the details would be the zodb list on google groups, https://groups.google.com/forum/#!forum/zodb

[tseaver]

Given how packing works, I'm doubtful that it would be reasonable to implement the "log packed-away records" feature during packing: there is no easily-accessible record of the transactions which are being packed away.

Instead, perhaps the feature could be based on a periodic script which logs all records between and ; one could then use the same to drive the pack. Such a script could use the same ZODB API methods as the Zope "Undo" tab.

[idgserpro]

I'm doubtful that it would be reasonable to implement the "log packed-away records" feature during packing: there is no easily-accessible record of the transactions which are being packed away.

@tseaver This is a reasonable explanation of why it's not possible to have this feature as described in this issue. We're not so familiar with ZODB sourcecode to know these limitations (we're asking for this feature from a high level perspective) and that's why we started this discussion. If not asking too much, would you have a little more complete explanation of this limitation, if possible with some source code references?

Instead, perhaps the feature could be based on a periodic script which logs all records between and ; one could then use the same to drive the pack. Such a script could use the same ZODB API methods as the Zope "Undo" tab.

We can indeed log all records: in fact, third party plugins in Plone world do exactly that, but using the Zope event machinery instead of interacting directly with ZODB. The plugin "solves" the issue, but the idea here is to have this log file generated directly from ZODB, out-of-the-box, since it's already available using the undoInfo information. And this info would be useful not only to Zope/Plone users but to everyone that uses ZODB (no need to create a plugin that hooks to events in each framework). This seemed as a simpler and more reasonable solution than hooking to a a lot of events but now you're saying it's more complicated than it looks.

That's perfectly ok: having this technical information, at least we can point this issue to clients when we're confronted with "why we can't have this undo information on disk".

Thanks for stepping by.