AMI set, Process immediately v/s enqueue, Questions about ami

[giancarlobi]

@DiegoPino I think it's a good idea a dedicated queue for ingest AMI specific set ID. In archipelago philosophy we have to manage that as simple as possible, without confusing user so:

• an AMI item can be added only to one queue, specific or hydroponics and with only one flag for the user to be clear
• hydroponics and immediate queue are to be completely independent regarding execution and code and everything
• immediate queues are to be one at time because if you want parallel processing then use hydroponics
• I think undo queue is a complexity that we don't need and use node revision could be enough. As extreme idea could be an ADO export (does AMI allow this?) of current ADO before ingesting, but not sure this make sense.

[giancarlobi]

AMI export: That would be nice. I guess I could do that using a very simple approach on a spreadsheet where each column is a JSON key and the values actually JSON snippets? I will think about that

@DiegoPino I was thinking in AMI export based on my current workflow in I7 to manage DC:

• a script extract DC from a collection and create a CSV file with DC elements as title of the columns
• librarians edit CSV file adding/editing DC entries (a row for each volume)
• librarians return CSV and a second script generate single DC.xml
• each DC xml are ingested by drush update command

Just in case for some more ideas.

[DiegoPino]

In the same tone of this:

I might want to acquire a Drupal Lock when immediate Processing is requested.

 $lock = Drupal::lock();
 if ($lock->acquire('ami_set_process_1', 10.0)) { // 10 seconds per ADO, this sounds good since a single one is not taking more than 2 seconds, but this may vary?
        // Batch process, keep renewing the Lock while its running   
}

[DiegoPino]

@giancarlobi as always thanks for your feedback. Yes, consistency is the key. Yes, the flag is one only in the new pull. I feel my wording there is confusing (the fact that checkbox checked means "enqueue" instead of "process immediately"? But still its a single one
Yes. Also, immediate queue are not parallel (at least in the sense of Per Set, per User, since really multiple users could process multiple batches via the UI, not sure I can control that other than establishing Drupal Locks).

The undo queue may only make sense if Reverting a revision is not automatized. I feel that is something I can do, but only for Update/Patch operations.

AMI export: That would be nice. I guess I could do that using a very simple approach on a spreadsheet where each column is a JSON key and the values actually JSON snippets? I will think about that

[DiegoPino]

Yes! We want the "roundtrip" experience. Factor I may need to solve is the hierarchical JSON in a cell.
Once way of dealing with that is allowing a single object to span to multiple Rows. That requires, to make things for the ingester easier, also a ROW_NUM column, since the parent/child relationship would be easier to manage in that case by not using the actual ROW number (which will be shifted every time you add a new value under a Column).

Sadly this function here I wrote is rocket science and I always fear I will break everything if I change it. That one is that builds the hierarchy, checks if UUIDs are valid and parents present and orders the ingest queue from parents first to children later. If not using (or allowing as alternative) a ROW ID column the change will just need to happen there.

[DiegoPino]

Resolved for now via ee93a15
I will use the feedback here for other ISSUEs and the next steps. Thanks people