Comments (6)
@DiegoPino I think it's a good idea a dedicated queue for ingest AMI specific set ID. In archipelago philosophy we have to manage that as simple as possible, without confusing user so:
- an AMI item can be added only to one queue, specific or hydroponics and with only one flag for the user to be clear
- hydroponics and immediate queue are to be completely independent regarding execution and code and everything
- immediate queues are to be one at time because if you want parallel processing then use hydroponics
- I think undo queue is a complexity that we don't need and use node revision could be enough. As extreme idea could be an ADO export (does AMI allow this?) of current ADO before ingesting, but not sure this make sense.
from ami.
AMI export: That would be nice. I guess I could do that using a very simple approach on a spreadsheet where each column is a JSON key and the values actually JSON snippets? I will think about that
@DiegoPino I was thinking in AMI export based on my current workflow in I7 to manage DC:
- a script extract DC from a collection and create a CSV file with DC elements as title of the columns
- librarians edit CSV file adding/editing DC entries (a row for each volume)
- librarians return CSV and a second script generate single DC.xml
- each DC xml are ingested by drush update command
Just in case for some more ideas.
from ami.
In the same tone of this:
I might want to acquire a Drupal Lock when immediate Processing is requested.
$lock = Drupal::lock();
if ($lock->acquire('ami_set_process_1', 10.0)) { // 10 seconds per ADO, this sounds good since a single one is not taking more than 2 seconds, but this may vary?
// Batch process, keep renewing the Lock while its running
}
from ami.
@giancarlobi as always thanks for your feedback. Yes, consistency is the key. Yes, the flag is one only in the new pull. I feel my wording there is confusing (the fact that checkbox checked means "enqueue" instead of "process immediately"? But still its a single one
Yes. Also, immediate queue are not parallel (at least in the sense of Per Set, per User, since really multiple users could process multiple batches via the UI, not sure I can control that other than establishing Drupal Locks).
The undo queue may only make sense if Reverting a revision is not automatized. I feel that is something I can do, but only for Update/Patch operations.
AMI export: That would be nice. I guess I could do that using a very simple approach on a spreadsheet where each column is a JSON key and the values actually JSON snippets? I will think about that
from ami.
Yes! We want the "roundtrip" experience. Factor I may need to solve is the hierarchical JSON in a cell.
Once way of dealing with that is allowing a single object to span to multiple Rows. That requires, to make things for the ingester easier, also a ROW_NUM column, since the parent/child relationship would be easier to manage in that case by not using the actual ROW number (which will be shifted every time you add a new value under a Column).
Sadly this function here I wrote is rocket science and I always fear I will break everything if I change it. That one is that builds the hierarchy, checks if UUIDs are valid and parents present and orders the ingest queue from parents first to children later. If not using (or allowing as alternative) a ROW ID column the change will just need to happen there.
from ami.
Resolved for now via ee93a15
I will use the feedback here for other ISSUEs and the next steps. Thanks people
from ami.
Related Issues (20)
- Undefined array key "#description_display" in Ami Row Preview Controller HOT 3
- Cap the File URL to 255 chars HOT 2
- VBO facets can't run on an anonymous session because of Private Store HOT 1
- AMI Set Validation HOT 1
- AMI Sets Analysis Overview
- Ensure ZIP sanity before trying to access it
- Autogenerate sequence_id for `ispartof` as an Admin Option HOT 1
- Allow multi ADO relationships on AMI CSVs HOT 1
- Add alternate mimetype mappings HOT 2
- CSV Column/Header Sort HOT 4
- Improve/fix AMI set Access Control/Permissions for AMI Set lists, Operations, etc HOT 4
- Add an additional data_lod twig context that includes the original label used to reconciliate against. HOT 1
- Ingest Failsafe on low Space HOT 1
- Owning user deletion and errors on default formatters (and others)
- Owning user deletion and errors on default formatters (and others) HOT 1
- Clean leading dots from remote downloaded files
- AMI CSV export fails when an ADO has a missing `type` key HOT 1
- Modern JS (latest 112.0.5615.121 (Official Build) (x86_64)) and our wonky #states bug HOT 1
- Remove check for getMasterRequest() HOT 1
- Provide Twig variables for Ajax Preview in Twig templates
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ami.