User story

As a user, I want to run a method to easily update custom metadata.

Acceptance criteria

• When a user runs a method with arguments with a supplied set of data asset IDs, these assets will have modality, subject id, platform, collection date, institution, and data level filled out correctly.
• The values for these fields should come from the document db API.
• There is an option to print out the changes to a log file (default to True)
• Appropriate docstrings and unit tests

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

Add any helpful notes here.

Is your feature request related to a problem? Please describe.
When trying to use CodeOceanJob to handle metadata tags correctly, I spent couple hours looking at the control flow, and I'm still not sure I get it. Some other inconsistencies were also confusing - wrappers around configuration objects in aind-codeocean-api that really were only renaming things. We should also support a reprocessing workflow - registration should be optional.

Describe the solution you'd like
I propose CodeOceanJob be organized as follows:

• Use aind-codeocean-api's request/configuration objects directly for the Register, Run, and Capture steps.
• Add a pass_metadata_to_result flag, default True
• Add a add_data_level_tags flag, default True
• Ensure that all metadata tags are unique (use Set, rather than List)
• Make sure that Run+Capture can be used independently of Register+Run+Capture

Is your feature request related to a problem? Please describe.
Currently, the aind-trigger-codeocean has some classes to run specific capsules, register assets, and capture results (see: https://github.com/AllenNeuralDynamics/aind-trigger-codeocean/blob/main/code/aind_trigger_codeocean/pipelines.py#L104)
However, such class should live here and the aind-trigger-codeocean repo should be a CO capsule to trigger jobs using this class.

Describe the solution you'd like
Ideally, a CodeOceanJob class should:

1. register an asset (if needed) or mount a registered asset
2. run a capsule/pipeline from an ID
3. optionally wait
4. optionally capture results
5. optionally send Teams notifications

Describe alternatives you've considered
One could use the aind-codeocean-api directly, but registration and waiting for results to capture requires some additional and non-trivial coding

User story

As a user, I want the code published to PyPI, so I can easily install it in other packages.

Acceptance criteria

• When code is merged into main, then the package is published to PyPI

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

Add any helpful notes here.

Move bot alert from aind-trigger-codeocean here

Is your feature request related to a problem? Please describe.
We have a large number of assets that users have archived. These are using unnecessary space and cost.

Describe the solution you'd like
A method that let's me see a list of all archived data assets older than a particular age and then separately decide to delete them. I should optionally be able to exclude assets that have attachements.

User story

As a user, I'd like to use a map to replace tags, to make it easier to replace tags instead of running remove and add separately.

Acceptance criteria

• Given a user calls update_tags, then they can supply an arg tags_to_replace: Optional[Dict[str,str]] = None that will change tags in the data_assets list.

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

Add any helpful notes here.

User story

As an admin, I would like to know which Runs we could potentially delete so as to save space/cost.

Code Ocean Capsules and Pipelines have many Runs that store output files. In many cases these runs are part of the normal testing cycle and can be removed.

Acceptance criteria

• A script we can run regularly that produces a CSV with columns: capsule name, run datetime, size

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

This script should not actually delete data - that will be a separate task.

*Original issue:

Code Ocean Capsules and Pipelines have many Runs that store output files. In many cases these runs are part of the normal testing cycle and can be removed.

Write a script uses the Code Ocean API to identify Runs that we could potentially delete. The script should output a CSV with the following columns:
1. Capsule name
2. Run date / time
3. Whether the Run was captured as a Data Asset
4. Total size of files

We should be able to generate this report whenever we like. Actually deleting data will be a separate task.

User story

As a user, I want a method to update tags on assets, so I can easily update tags.

Acceptance criteria

• When a user runs a method with arguments old_tag and new_tag and iterator of data asset ids, then all assets with those data asset ids in code ocean and have old_tag will have "old_tag" changed to new_tag.
• If old_tag is None, then all assets satisfying the filter will be tagged with new_tag.
• There is an option to print out the changes to a log file (default to True)
• Appropriate docstrings and unit tests

Sprint Ready Checklist

• 1. Acceptance criteria defined
• 2. Team understands acceptance criteria
• 3. Team has defined solution / steps to satisfy acceptance criteria
• 4. Acceptance criteria is verifiable / testable
• 5. External / 3rd Party dependencies identified
• 6. Ticket is prioritized and sized

Notes

Add any helpful notes here.

Is your feature request related to a problem? Please describe.
Assets are not being tagged automatically, which is making them difficult to find.

Describe the solution you'd like
raw data should be tagged with the DataLevel.RAW tag and derived data should be tagged with the DataLevel.DERIVED tag.