canonical / charming-actions Goto Github PK

Bug Description

We recently added the canonical/charming-actions/release-libraries action to our reusable workflows and have isolated an issue we are seeing to the inclusion of this action in the reusable workflow. With this job:

  release-charm-libs:
    name: Release charm libs
    uses: canonical/charming-actions/[email protected]
    needs: [ lib-check, tests, integration-tests ]
    with:
      credentials: ${{ secrets.CHARMHUB_TOKEN }}
      github-token: ${{ secrets.GITHUB_TOKEN }}

To https://github.com/canonical/operator-workflows/blob/main/.github/workflows/test_and_publish_charm.yaml (lines 90-96), we get the following failure:

[Invalid workflow file: .github/workflows/test_and_publish_charm.yaml#L19](https://github.com/canonical/indico-operator/actions/runs/4000676362/workflow)
error parsing called workflow
".github/workflows/test_and_publish_charm.yaml"
-> "canonical/operator-workflows/.github/workflows/test_and_publish_charm.yaml@main" (source branch with sha:a914f5ce827f00cc5e314b2d54ff9227a79c95ff)
: invalid value workflow reference: references to workflows must be rooted in '.github/workflows'

Without it, the workflow runs fine. Obviously the above error is a red herring, I'm not sure what is causing the problem. Do you have any ideas?

To Reproduce

Add:

  release-charm-libs:
    name: Release charm libs
    uses: canonical/charming-actions/[email protected]
    needs: [ lib-check, tests, integration-tests ]
    with:
      credentials: ${{ secrets.CHARMHUB_TOKEN }}
      github-token: ${{ secrets.GITHUB_TOKEN }}

To a reusable workflow and call it from a repository

Environment

This is running on GitHub actions

Relevant log output

[Invalid workflow file: .github/workflows/test_and_publish_charm.yaml#L19](https://github.com/canonical/indico-operator/actions/runs/4000676362/workflow)
error parsing called workflow
".github/workflows/test_and_publish_charm.yaml"
-> "canonical/operator-workflows/.github/workflows/test_and_publish_charm.yaml@main" (source branch with sha:a914f5ce827f00cc5e314b2d54ff9227a79c95ff)
: invalid value workflow reference: references to workflows must be rooted in '.github/workflows'

### Additional context

_No response_

This charm has logic to skip publishing when called from branches/events that do not meet these conditions. If called from a non-publishable event, the action returns as passing (green in github actions page) even though nothing was published.

When CI tests this charm in this repo, it is subject to the same conditions as above. Thus, testing a PR from a branch other than the allowed names (eg: branch/my-branch) results in the CI showing green but not having been run. This should be changed

Possible fixes:

• add an if statement on the tests so we clearly see when they've been skipped vs run
• add checks in CI to ensure we're calling from a branch that will test, otherwise fail
• add an override option to the branch logic so we can force the publishing to always occur
• ?

If we don't do something that's really obvious, we should also add notes in the readme for developers (eg: when making mods to this repo, do them from branches called branch/my-new-feature)

Enhancement Proposal

One of the biggest bottlenecks of charm development is building charm sin the CI: it takes forever, and it only gets worse as the list of charms to build grows. The data-platform-workflows (by @carlcsaposs-canonical) repository contains a series of GH actions that can be leveraged for building the charms with a cache and reuse them across different jobs in a workflow.

The most obvious example where this is useful is the publish job, which also packs the charm(s) in the repo on the same CI where other jobs are running, but there could be other actions that may be benefited from this.

For more information, go here

Bug Description

Running charmcraft 2.0.0, the upload-charm action published this release with undefined version number.

I believe the source of this issue is because the --quiet flag's behaviour has changed:

• charmcraft 1.7.x: --quiet removed all output except for the final message ("Revision X ..." for success, or "Upload failed with status ..." for failure)
• charmcraft 2.0.0: --quiet silences all output, both for success and failure. So revision cannot be pulled from the stdout, and an error cannot be inferred from the text either (but can be inferred from $?)

To Reproduce

see this ci run

You can also try yourself by charmcraft upload somecharm --quiet on 1.7.x and 2.0.0

Environment

charmcraft 2.0.0

Relevant log output

see [this ci run](https://github.com/canonical/alertmanager-k8s-operator/actions/runs/2676384442)

Additional context

Fixing this so it properly parses charmcraft 2.0.0 would be excellent, but we should also add checks that would have marked rev=undefined as a failure in the action as well. That way it fails loudly, rather than "works" incorrectly

Bug Description

Github has deprecated the save-state and set-output commands. Using these commands will now notify with a deprecation message, and in future not work at all. There is a new method available for the same effect, as described in the linked post.

The following items need to be updated to fix this:

• javascript package
  • channel.ts
  • check-libraries.ts
• printing to screen
  • get_charm_paths.py

To Reproduce

N/A

Environment

N/A

Relevant log output

N/A

Additional context

No response

this messaage shows when, for example, the channel action is used on a PR that is against a feature branch (eg: PR from some-branch to some-other-branch, where the target is not the default branch or a track/* branch). This message does not explain what is happening in a way someone without extra context could interpret.

Enhancement Proposal

The current logdump action is not very good and has quite a lot of annoying limitations. We should switch it out to an action in here, powered by https://github.com/juju/juju-crashdump.

Bug Description

It is a minor report, just to keep the house clean.
GitHub reports warning:

Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: canonical/charming-actions/[email protected]. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

You can find it here: https://github.com/canonical/s3-integrator/actions/runs/4004942011

To Reproduce

Use "charming-actions/[email protected]" is GH CI/CD and check Actions->Summary->Warnings.

Environment

GH Action with the latest availabe [email protected]

Relevant log output

Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: canonical/charming-actions/[email protected]. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

Additional context

The warning is obliviously reported for other actions too, e.g. channel:

Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/checkout@v2, canonical/charming-actions/[email protected], canonical/charming-actions/[email protected]. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.

in https://github.com/canonical/mysql-operator/actions/runs/3986561030

Based on the readme.md files in the root and /channel, it is undefined what CI outcomes are expected when a push/PR is out of scope for the channel action. channel/readme.md notes that pushes to and PRs against any branch other than default or track/* are Ignored, but does not state whether an error is thrown or if a value is returned in steps.<ID>.outputs.name in that event. Looking at the code I believe an error is thrown, but my javascript is poor :)

If errors are thrown for out-of-scope channels, is that the desired outcome? A failure shows as an :X: in the CI (and will block merges in github if repos have that enabled). I think we either need to run successfully even when this is an out-of-scope channel (maybe name="" is defined as the return for out-of-scope events?), or in our workflow snippet in the root readme.md we need to advise on how to handle these events in a way that doesn't fail the CI. If we return name="", we'd also need an if statement for the upload-action so we skipped it if name was empty (otherwise either upload-charm will fail or will send to the default channel, not sure how github would handle that)

However we handle failures for out-of-scope channels, I think we need to set up the workflow so it gives a clear message of what happened in the CI/checks summary. I'd expect outcomes to be:

• PR/push against default/track: CI for publish is ✔️ if it uploaded everything successfully, :X: if it failed to upload
• PR/push against out-of-scope: CI for publish is... skipped symbol (eg: did not run, but is not an error)
  That might require splitting the suggested workflow up into two jobs: Set destination channel and Publish to selected channel (or better names :) ). Then Publish to selected channel will depend on (need) the Set destination channel, and the job can run only if Set destination channel was successful (or, if it returned name != ""). I think that would then achieve a clear picture in CI, where the publish will only show :X: if it actually broke, rather than just should not have ran

The Kubeflow team had CI using this action recently fail when executed from a fork. We believe this is because the fork did not define the credentials secret, thus meaning the credentials argument passed to upload-charm was an empty string.

Assuming this is the correct failure mode, upload-charm should catch and fail with an obvious error when it is passed blank or obviously incorrect credentials. Calling the action with an empty credential is an especially easy error to make in github actions, as actions will silently use an uninstantiated variable rather than raise an error.

Originally posted by @simskij in #21 (comment)

As long as the file resource in question is available in the charm root folder, uploading file resources should be a quite trivial addition. To be clear, I don't think the action needs to be able to download the resource itself - that could be done in a prior step, just upload them to charmhub and add them to the release.

https://github.com/canonical/charmhub-upload-action/blob/27ccba841400bb79f38acd36e63b67de2db051fd/main.js#L95-L97

Enhancement Proposal

The code here uploads the single charm only.
Hovewer the charmcraft.yaml supports multiple bases (which is critical for subordinate operators).

It is an enhancement request to improve 'upload-charm' action to upload all *.charm files produced by charmcraft.
Thank you in advance!

Bug Description

Errors on upload-charm:

Error: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
Error: HttpError: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}

To Reproduce

Rerun this job:
https://github.com/canonical/alertmanager-k8s-operator/runs/7455137256?check_suite_focus=true

Environment

GH runner.

Relevant log output

Run canonical/charming-actions/[email protected]
/usr/bin/sudo snap install charmcraft --classic --channel latest/edge
charmcraft (edge) 2.0.0+6.g7563ce5 from Canonical** installed
/snap/bin/charmcraft pack --destructive-mode --quiet
/usr/bin/docker pull ubuntu/prometheus-alertmanager:0.23-22.04_beta
0.23-22.04_beta: Pulling from ubuntu/prometheus-alertmanager
bdc5a07a7dd2: Pulling fs layer
aa9fa10e633d: Pulling fs layer
e5dd500af9cc: Pulling fs layer
676141a7b9c0: Pulling fs layer
3c2910da3d19: Pulling fs layer
5fdd3cbd15aa: Pulling fs layer
78b2bc159136: Pulling fs layer
3c2910da3d19: Waiting
5fdd3cbd15aa: Waiting
78b2bc159136: Waiting
676141a7b9c0: Waiting
e5dd500af9cc: Verifying Checksum
e5dd500af9cc: Download complete
bdc5a07a7dd2: Verifying Checksum
bdc5a07a7dd2: Download complete
3c2910da3d19: Verifying Checksum
3c2910da3d19: Download complete
676141a7b9c0: Verifying Checksum
676141a7b9c0: Download complete
78b2bc159136: Verifying Checksum
78b2bc159136: Download complete
5fdd3cbd15aa: Verifying Checksum
5fdd3cbd15aa: Download complete
aa9fa10e633d: Verifying Checksum
aa9fa10e633d: Download complete
bdc5a07a7dd2: Pull complete
aa9fa10e633d: Pull complete
e5dd500af9cc: Pull complete
676141a7b9c0: Pull complete
3c2910da3d19: Pull complete
5fdd3cbd15aa: Pull complete
78b2bc159136: Pull complete
Digest: sha256:8f30b70d39053bf979ba646c1444287d71f7ece2342dea518106a6077197cd15
Status: Downloaded newer image for ubuntu/prometheus-alertmanager:0.23-22.04_beta
docker.io/ubuntu/prometheus-alertmanager:0.23-22.04_beta
/snap/bin/charmcraft upload-resource --quiet alertmanager-k8s alertmanager-image --image ubuntu/prometheus-alertmanager:0.23-22.04_beta
/snap/bin/charmcraft resource-revisions alertmanager-k8s alertmanager-image
Revision    Created at              Size
15          2022-07-21T18:08:50Z    515B
14          2022-07-20T15:27:02Z    515B
13          2022-07-15T11:48:59Z    515B
12          2022-07-06T15:54:06Z    515B
11          2022-07-05T23:27:14Z    515B
10          2022-07-03T20:54:42Z    515B
9           2022-06-30T09:06:59Z    515B
8           2022-06-28T03:41:22Z    515B
7           2022-05-30T21:35:04Z    515B
6           2022-04-28T20:38:02Z    515B
5           2022-03-24T14:04:40Z    515B
4           2022-03-22T15:37:00Z    515B
3           2022-03-17T20:09:33Z    515B
2           2022-03-16T21:47:46Z    515B
1           2021-07-21T18:28:17Z    515B
/snap/bin/charmcraft upload --quiet --release latest/edge /home/runner/work/alertmanager-k8s-operator/alertmanager-k8s-operator/alertmanager-k8s_ubuntu-20.04-amd64.charm --resource=alertmanager-image:15
Error: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
Error: HttpError: Validation Failed: {"resource":"Release","code":"already_exists","field":"tag_name"}
    at /home/runner/work/_actions/canonical/charming-actions/1.0.0/dist/upload-charm/index.js:7440:21
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
Total size of all the files uploaded is 9165 bytes
Finished uploading artifact charmcraft-logs. Reported size is 9165 bytes. There were 0 items that failed to upload
Artifact upload result: {"artifactName":"charmcraft-logs","artifactItems":["/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-175635.208482.log","/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-180804.853589.log","/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-180853.491465.log","/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220721-180855.218908.log"],"size":9165,"failedItems":[]}

Additional context

No response

Publishing a charm specifying an uptream resource hosted on a private repository that requires authentication is not currently supported. Specifically, I'm trying to use an image uploaded to charmhub

upstream-source: 'registry.jujucharms.com/charm/e7cgxprwp5telxhlb24qmbxuoi9a5e3m40iv/indico-image@sha256:c40b5291a0346132d1f6ed5a38f92fafdefd7da64655742881729a8cbaf6a807'

getting the following error message:

Error response from daemon: pull access denied for undefined, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Ideally the action should adjust its output based on the differences between the repo and charmhub:

Just a note for @DomFleischmann that we need to think
carefully about using this, and what the succession plan is > for that tool now that Ken has likely stopped maintaining it.

Originally posted by @jnsgruk in #21 (comment)

Enhancement Proposal

When running in a multi-charm repo like kfp-operators, we execute check-libraries multiple times in the same CI run. When one of those runs returns with a library needing a version bump, it is unclear from the message which one it is.

It would be great if the returned message included the path that the check-libraries function was run on.

Bug Description

The upload-charm action fails when it tries to upload any new resource. For example, if you have an existing charm and add a resource to that charm, you must manually upload an instance of that resource before the action will work. It would be good if the action caught this and uploaded the resource for you as expected.

To Reproduce

Add a resource to a charm, then try to upload the charm using this action

Environment

github action run for any environment

Relevant log output

Logs from a past action run, which attempted to upload resource `myfile` for the first time: 

Error: Resource 'myfile' does not have any uploaded revisions.
Error: Error: Resource 'myfile' does not have any uploaded revisions.
    at Charmcraft.<anonymous> (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc/dist/upload-charm/index.js:21737:23)
    at Generator.next (<anonymous>)
    at fulfilled (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc/dist/upload-charm/index.js:21633:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Additional context

No response

This uses charmcraft resource-revisions to get all revisions, then takes the top line as the resource we just uploaded. This will often be true, but might not be if:

• charmcraft ever starts noticing we don't need to upload everything, every time and instead reuses old revisions
• (unlikely, but also really hard to debug) multiple publishes happen at once and the resource we pushed gets bumped down in the order

Suggested fix: parse revision number from the output from charmcraft upload-resource which says something like Revision N created of resource ....

Bug Description

Using the following upload-bundle action fails with the following error: Parts processing error: Failed to copy '/tmp/.tmpDpiw3w/build/stage/icon.svg': no such file or directory. But the repository [1] has an icon.svg file, and regular charmcraft pack works.

[1] https://github.com/claudiubelu/waltz-juju-bundle

To Reproduce

Github action:

      - name: Upload bundle to edge
        uses: canonical/charming-actions/[email protected]
        with:
          credentials: "${{ secrets.CHARMCRAFT_AUTH }}"
          github-token: "${{ secrets.GITHUB_TOKEN }}"
          channel: "${{ steps.release-short.outputs.short_version }}/edge"

Environment

Github action: https://github.com/claudiubelu/waltz-juju-bundle/blob/560951f1982fdc6bc37829186c40681d22adfced/.github/workflows/create_release.yaml

Relevant log output

Error: Error running subcommand ``charmcraft pack -p /tmp/.tmpDpiw3w``: `Packing the bundle.                                                            
Parts processing error: Failed to copy '/tmp/.tmpDpiw3w/build/stage/icon.svg': no such file or directory.                                                      
Full execution log: '/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220416-011600.638784.log'       

Failure: https://github.com/claudiubelu/waltz-juju-bundle/runs/6044583147?check_suite_focus=true

### Additional context

_No response_

Bug Description

For tracks that have charmhub branches, such as:

charmcraft status metacontroller-operator                                
Track    Base                  Channel                    Version    Revision    Expires at                                                                                             
latest   ubuntu 20.04 (amd64)  stable                     2          2                                                                                                                  
                               candidate                  5          5                                                                                                                  
                               beta                       ↑          ↑                                                                                                                  
                               edge                       13         13                                                                                                                 
                               edge/add-charming-actions  9          9           2022-06-30T01:00:00Z                                                                                   
                               edge/resources-removal     12         12          2022-07-09T01:00:00Z                   

the release-charm action fails due to incorrect parsing of the available channels.

To Reproduce

See this action run

Environment

See this action run

Relevant log output

See [this action run](https://github.com/canonical/metacontroller-operator/runs/7073697720?check_suite_focus=true)

Additional context

charmcraft edge now allows for structured output using charmcraft status charmName --format json. This would let us avoid parsing the human readable text. There are some growing-pains with this output, however.

If a charm has dependencies defined in its charmcraft.yaml file, it will try to apt-get install them. However, the upload-charm is hitting a Permission Denied error, which can be solved by just running sudo apt-get install instead.

sample build-packages in charmcraft.yaml:

parts:
  charm:
    build-packages:
      - git
      - python3-dev
      - libffi-dev
      - cargo
      - libssl-dev

Charm: https://github.com/claudiubelu/legend-juju-sdlc-server-operator
Job: https://github.com/claudiubelu/legend-juju-sdlc-server-operator/runs/5589471448?check_suite_focus=true
Logs: https://paste.ubuntu.com/p/pDgzgQMNm6/

Bug Description

The label "Libraries: Out of sync" is remain for PR with updated libs.

To Reproduce

1. create PR for the repo with outdated libs, repo got label "Libraries: Out of sync"
2. update libs in the same PR, check-libraries is re-triggered, but label remain "Libraries: Out of sync"

Expected results: label replaced from "Libraries: Out of sync" to "Libraries: OK"

Proposal: update the label if test result status != GitHub label.

Environment

PR: canonical/postgresql-k8s-operator#86
Test where old label remain: https://github.com/canonical/postgresql-k8s-operator/actions/runs/4024638068/jobs/6916898343

Relevant log output

Run canonical/charming-actions/[email protected]
  with:
    credentials: ***
    github-token: ***
    charm-path: .
    charmcraft-channel: latest/stable
    use-labels: true
    label-success: Libraries: OK
    label-fail: Libraries: Out of sync
    comment-on-pr: false
    fail-build: false
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.2.0 from Canonical** installed
/snap/bin/charmcraft fetch-lib
Library charms.data_platform_libs.v0.database_provides was already up to date in version 0.2.
Library charms.observability_libs.v1.kubernetes_service_patch was already up to date in version 1.5.
Library charms.postgresql_k8s.v0.postgresql was already up to date in version 0.7.
Library charms.postgresql_k8s.v0.postgresql_tls was already up to date in version 0.5.
Library charms.rolling_ops.v0.rollingops was already up to date in version 0.2.
Library charms.tls_certificates_interface.v1.tls_certificates was already up to date in version 1.12.
/usr/bin/git checkout HEAD -- lib

### Additional context

_No response_

As stated in the title, the release-charm action currently fails for resourceless charms. The reason seems to be that we're not guarding against empty resource blocks in the charmcraft output. This should solve itself when we move over to using the programmatic output, but we still need to track it here.

Examples

• https://github.com/canonical/prometheus-scrape-config-k8s-operator/runs/7726893672?check_suite_focus=true
• https://github.com/canonical/prometheus-scrape-target-k8s-operator/runs/7726964781?check_suite_focus=true

Likely problem

(cc @agathanatasha)

Nice work so far! 🚀

It would be great if this action could also detect changes in lib/charms/<charm_name>/v#/<lib_name> and publish updated version of charm libs. Thinking out loud, there are some gotchas:

• Need to ensure we only attempt to publish libraries for the charm in the repo, not those fetched for consumption
• Need to ensure that the LIBPATCH has actually been bumped and fail gracefully/warn if not

Tagging in @petevg and @simskij for the discussion too.

Enhancement Proposal

It would be handy if there was a dependabot-like scheduled action to automatically open a PR when fetch-lib results in an update.

For the observability bundle, we periodically open "upkeep" PRs manually for this purpose.

Bug Description

Upload-charm action fails when charm is updated from not having any resources to having a resource. When uploading the resources, charmcraft is verifying resources specification against the latest metadata on charmcraft (which is outdated in this scenario). The new charm needs to be uploaded first before uploading a resource.

To Reproduce

1. A new charm or a charm with no resources specified previously
2. Add resources specifications in metadata.yaml
3. Run upload-charm action

Environment

On github
failed github action run for reference

Relevant log output

/snap/bin/charmcraft upload-resource --quiet kratos oci-image --image 9f4e08abbf6a
Store operation failed:
- resource-not-found: No oci-image type "oci-image" resource found

Additional context

No response

If a lib version / libpatch is updated, I'd like an action to publish it to charmhub.

If you could add me to the contributors and assign this to me, I'll open a PR soon :)

Bug Description

HI,

The release-libraries silently keeps library non-published if LIBPATCH is too high due to human mistake
(e.g. the latest published LIBPATCH is 3 and the currently committed is 5):

> charmcraft publish-lib charms.mongodb.v0.mongodb_tls
Library charms.mongodb.v0.mongodb_tls has a wrong LIBPATCH number, 
it's too high and needs to be consecutive, Charmhub highest version is 0.3.                                                                                                                                    

> echo $?
0

Charmhub enforces LIBPATCH needs to be consecutive. Unfortunately exit code there is zero, so it is silently missing.

To Reproduce

1. bump LIBPATCH on 2+ versions to the current
2. try to publish new library

Environment

charmhub 2.2.0
charming-actions 2.2.2

Relevant log output

> charmcraft publish-lib charms.mongodb.v0.mongodb_tls
Library charms.mongodb.v0.mongodb_tls has a wrong LIBPATCH number, it's too high and needs to be consecutive, Charmhub highest version is 0.3.                                                                                                                                    

> echo $?
0

Additional context

No response

Kubeflow charms also have microk8s.inspect which could be handy.

    - name: Generate inspect tarball
      run: >
        sg microk8s <<EOF
          microk8s inspect | \
          grep -Po "Report tarball is at \K.+" | \
          xargs -I {} cp {} inspection-report-${{ strategy.job-index }}.tar.gz
        EOF
      if: failure()

    - name: Upload inspect tarball
      uses: actions/upload-artifact@v2
      with:
        name: inspection-reports
        path: ./inspection-report-${{ strategy.job-index }}.tar.gz
      if: failure()

Bug Description

When executing the the upload-action for a charm specifying an upstream-resource the exception attached occurs. This happens independently of the uploadand resource-overridesoptions specified:

- name: Upload charm to charmhub
        uses: canonical/charming-actions/[email protected]
        with:
          credentials: ${{ secrets.CHARMHUB_TOKEN }}
          github-token: ${{ secrets.GITHUB_TOKEN }}
          channel: edge
          upload: false

The resources section of the charm metadata file is the following:

resources:
  indico-image:
    type: oci-image
    description: Docker image for Indico
    auto-fetch: true
  indico-nginx-image:
    type: oci-image
    description: Docker image for nginx Indico
    auto-fetch: true
  nginx-prometheus-exporter-image:
    type: oci-image
    description: Prometheus exporter for nginx
    auto-fetch: true
    upstream-source: nginx/nginx-prometheus-exporter:0.10.0

To Reproduce

Run the following workflow of the indico operator changing the existing metadata contents for

resources:
  indico-image:
    type: oci-image
    description: Docker image for Indico
    auto-fetch: true
  indico-nginx-image:
    type: oci-image
    description: Docker image for nginx Indico
    auto-fetch: true
  nginx-prometheus-exporter-image:
    type: oci-image
    description: Prometheus exporter for nginx
    auto-fetch: true
    upstream-source: nginx/nginx-prometheus-exporter:0.10.0

Environment

The issue popped-up while running a github workflow

Relevant log output

/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.0.0 from Canonical** installed
/usr/bin/sudo charmcraft pack --destructive-mode --quiet
/usr/bin/docker pull undefined
/usr/bin/docker pull nginx/nginx-prometheus-exporter:0.10.0
Using default tag: latest
***Error response from daemon: pull access denied for undefined, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Error: The process '/usr/bin/docker' failed with exit code 1
Error: Error: The process '/usr/bin/docker' failed with exit code 1
    at ExecState._setResult (/home/runner/work/_actions/canonical/charming-actions/1.0.3/dist/upload-charm/index.js:3276:25)
    at ExecState.CheckComplete (/home/runner/work/_actions/canonical/charming-actions/1.0.3/dist/upload-charm/index.js:3259:18)
    at ChildProcess.<anonymous> (/home/runner/work/_actions/canonical/charming-actions/1.0.3/dist/upload-charm/index.js:3153:27)
    at ChildProcess.emit (events.js:314:20)
    at maybeClose (internal/child_process.js:1022:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
No charmcraft logs generated, skipping artifact upload.***
0.10.0: Pulling from nginx/nginx-prometheus-exporter
6ca88092a[12](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:13)c: Pulling fs layer
aaea71c792be: Pulling fs layer
6ca88092a12c: Verifying Checksum
6ca88092a12c: Download complete
aaea71c792be: Verifying Checksum
aaea71c792be: Download complete
6ca88092a12c: Pull complete
aaea71c792be: Pull complete
Digest: sha256:1c6eba17d7768401ba1ed056101fc56ec3e3b5c82e508fdd8968fbcd96ec9574
Status: Downloaded newer image for nginx/nginx-prometheus-exporter:0.10.0
docker.io/nginx/nginx-prometheus-exporter:0.10.0
/snap/bin/charmcraft upload-resource --quiet indico nginx-prometheus-exporter-image --image nginx/nginx-prometheus-exporter:0.10.0
/snap/bin/charmcraft resource-revisions indico nginx-prometheus-exporter-image
Revision    Created at              Size
***5           2022-08-23T10:06:10Z    545B***
4           2022-08-22T09:51:04Z    545B
3           2022-08-19T15:10:27Z    545B
2           2022-08-19T[13](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:14):35:[32](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:33)Z    5[45](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:46)B
1           2022-08-19T13:32:26Z    5[45](https://github.com/canonical/indico-operator/runs/7970721141?check_suite_focus=true#step:3:46)B

Additional context

No response

Bug Description

get an issue in CI with unclear fail message:

Also tested with 2.0.0 stable

Run canonical/charming-actions/[email protected]
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.0.0 from Canonical** installed
/snap/bin/charmcraft fetch-lib
No keyring found to store or retrieve credentials from.
Full execution log: '/home/runner/snap/charmcraft/common/cache/charmcraft/log/charmcraft-20220907-085012.313435.log'
Error: The process '/snap/bin/charmcraft' failed with exit code 1
Error: Error: The process '/snap/bin/charmcraft' failed with exit code 1
    at ExecState._setResult (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc2/dist/check-libraries/index.js:3276:25)
    at ExecState.CheckComplete (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc2/dist/check-libraries/index.js:3259:18)
    at ChildProcess.<anonymous> (/home/runner/work/_actions/canonical/charming-actions/2.0.0-rc2/dist/check-libraries/index.js:3153:27)
    at ChildProcess.emit (events.js:314:20)
    at maybeClose (internal/child_process.js:1022:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)

To Reproduce

we run following pipe on https://github.com/canonical/grafana-k8s-operator

  lib-check:
    name: Check libraries
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Check libs
        uses: canonical/charming-actions/[email protected]
        with:
          credentials: "${{ secrets.CHARMHUB_TOKEN }}"
          github-token: "${{ secrets.GITHUB_TOKEN }}"

for example see checks in canonical/grafana-k8s-operator#118

Environment

Relevant log output

see the bug description

Additional context

No response

Interesting, Charmcraft seems to fail to upload a charmhub branch when we add a / in the git branch. That's something we should probably address.

Originally posted by @DomFleischmann in #29 (review)

Enhancement Proposal

This request is similar to #16, in the sense that it is looking to address a gap currently in the existing actions for charms with file resources.
The upload-charm readme contains the following,

When uploading a charm with file resources, the most recent resource will be attached to the release. If you want to use a new resource, you'll have to cut a new resource revision prior to running the action.

If I'm not mistaken this would mean a workflow that wants to upload a resource + a charm in one go would need to look something like the below. This feels like extra boilerplate and a lack of consistency between the manual steps and the action. My suggestion would be a separate action dedicated to uploading file resources, that would prevent any breaking changes to the upload-charm action.

      - uses: actions/download-artifact@master
        with:
          name: <snap-name>
      - name: Install charmcraft
        run: sudo snap install charmcraft --channel=2.x/stable
      - name: Publish Charm Resource
        run: charmcraft upload-resource <charm-name> <resource-name> --filepath ./<file-name>
        env:
          CHARMCRAFT_AUTH: "${{ secrets.CHARMHUB_TOKEN }}"
      - name: Upload charm to charmhub
        uses: canonical/charming-actions/[email protected]
        with:
          credentials: "${{ secrets.CHARMHUB_TOKEN }}"
          github-token: "${{ secrets.GITHUB_TOKEN }}"
          channel: "${{ steps.channel.outputs.name }}"

Interested to hear alternate approaches to this issue.

Rather than selectively uploading intermediate work to charmhub branches (eg: latest/edge/someBranch), we could upload all PR updates to charmhub. charmhub branches are ephemeral (ttl=30day) anyway so it won't make too much garbage, and this way we can easily test deployments.

There might be some logic that needs working through as we still need to know the destination track/channel that the branch should go to, but this could be the same as we currently use (if git branch is called /track/..., push to charmhub at /track/edge/branchName, else push to latest/edge/branchname)

if one works on a feature branch and wants to publish a charm, then the action will fail.
for example, I want to merge feat-999-add-relation-psql into feat-999 branch, then action will fail because Unhandled PR base name feat-999

while would be good to publish charm (according to the internal logic) to latest/edge/feat-999-add-relation-psql

real use case: https://github.com/canonical/mlflow-operator/actions/runs/4344924048/jobs/7588922599

Bug Description

Charmcraft's charmcraft upload-resource by design requires any image to be upload be specified by digest, not by image name. In charmcraft <=2.1.0, charmcraft upload-resource did not check for this and by chance used the input in a way that successfully uploaded the image even though it was not specified by digest.

Charmcraft 2.1.0 fixes a bug in handling image digests in a way that
breaks using tagged images (specifically, charmcraft treats any input with a : as a digest, like sha256:digest). This means the upload-charm action fails because it provides an image name instead of a digest.

To Reproduce

Use upload-action in any CI with charmcraft >2.1.0

Environment

Github runners

Relevant log output

Example of failure: https://github.com/canonical/kubeflow-profiles-operator/actions/runs/3235732172/jobs/5300571800

Additional context

No response

Enhancement Proposal

The charming actions v2.3.0 use deprecated node12. Github started enforcing migration to node16 (see this article). The main branch is already updated with the new version, but would it be possible to create a new tag for it?
With a new tag, renovate would open PRs to update repos which use the charming actions in CI.

Bug Description

When the major version increases and minor decreases, the library skips releasing the update.

See promethus-k8s-operator CI log output below. prometheus_remote_write 1.1 should have been published.

To Reproduce

Bump major version. Set minor to 0.

Environment

Github

Relevant log output

Run canonical/charming-actions/release-libraries@main
/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.4.1 from Canonical** installed
/snap/bin/charmcraft list-lib prometheus-k8s --format=json
[
    {
        "charm_name": "prometheus-k8s",
        "library_name": "prometheus_remote_write",
        "library_id": "f783823fa75f4b7880eb70f2077ec259",
        "api": 0,
        "patch": 15,
        "content_hash": "477f3dc99094dac025835a3691df3462ceeb360ee0ade59149bbcabcf08b5c88"
    },
    {
        "charm_name": "prometheus-k8s",
        "library_name": "prometheus_scrape",
        "library_id": "bc84295fef5f4049878f07b131968ee2",
        "api": 0,
        "patch": 39,
        "content_hash": "dd816a001c52ca853537e8a772f4548a090e7d441b126b71bbedc58e0d226c76"
    }
]
found lib file: ./lib/charms/prometheus_k8s/v0/prometheus_scrape.py. Parsing...
found lib file: ./lib/charms/prometheus_k8s/v1/prometheus_remote_write.py. Parsing...
checking status for prometheus_scrape
remote lib: {"libName":"prometheus_scrape","version":0,"revision":39}
prometheus_scrape will be updated from 0.39 to 0.40
checking status for prometheus_remote_write
remote lib: {"libName":"prometheus_remote_write","version":0,"revision":15}
prometheus_remote_write has a lower version or revision than prometheus_remote_write
Errors found during lib comparison.
Publishing changes:
[{"libName":"prometheus_scrape","old":{"major":0,"minor":39},"new":{"minor":40,"major":0}}]
publishing prometheus_scrape
Successfully published libraries
/snap/bin/charmcraft publish-lib charms.prometheus_k8s.v0.prometheus_scrape
Library charms.prometheus_k8s.v0.prometheus_scrape sent to the store with version 0.40

Additional context

No response

Enhancement Proposal

Hi,

At the moment upload-charm is always building charms using --destructive-mode for the speed reason.
Unfortunately it cause issues when GH runners (e.g. ubuntu-latest) are migrated to new versions and charms are still using old base (#1, #2). It would be great to avoid --destructive-mode if built-on doesn't match the runner's host OS version.

The proposal came from @jnsgruk here. Tnx!

P.S. it is not always possible to update base-on for subordinate charms as we need to support multiple serials in parallel.

Enhancement Proposal

It would be useful to return structured output when executing the upload-charm action including the uploaded charm revision and the charm resources revisions to be able to use them in future steps pf the workflow.

Thank you

This is probably a lot to ask for, but:
A revision is created when all I changed was tests, which are not included in the charm.
Parsing charmcraft.yaml and re-implementing charmcraft's logic is probably too much to ask for though.
Unless this could be turned into a bot with commands..? In that case we'd use commands such as $$merge$$ and $$merge-no-upload$$.

Every time we merge a PR to main or a track/foo branch a charm will be published to latest/edge or foo/edge in charmhub.

It is easy to know what code corresponds to the charm in edge as it will always be the latest code. But once we start promoting different revisions we will lose track of that.

Therefore, when publishing a new charm we should also tag the latest commit in our git history with the charm revision of the published charm. That way we can search for these revisions easily in our git history.

This should only apply for when we publish to a channel, not for a charmhub branch

Bug Description

Recently the snap library was bumped to v2 in canonical/operator-libs-linux#92.

The CI didn't seem to catch that, which you can see here

To Reproduce

Update a library LIBAPI version, moving the file into the new version

Environment

operator-libs-linux repo

Relevant log output

/usr/bin/sudo snap install charmcraft --classic --channel latest/stable
charmcraft 2.3.0 from Canonical** installed
/snap/bin/charmcraft list-lib operator-libs-linux --format=json
[
    {
        "charm_name": "operator-libs-linux",
        "library_name": "apt",
        "library_id": "7c3dbc9c2ad44a47bd6fcb25caa270e5",
        "api": 0,
        "patch": 11,
        "content_hash": "e16de26b23a6b35ef565c99d49d511f3e781b8d56a58a6f1edf2f4e0e3f6f3a4"
    },
    {
        "charm_name": "operator-libs-linux",
        "library_name": "dnf",
        "library_id": "1e93f444444d4a4a8df06c1c16b33aaf",
        "api": 0,
        "patch": 1,
        "content_hash": "c030c118a493f355fca303f07ca4795598f708d8931af0d9b4aa1eee92156880"
    },
    {
        "charm_name": "operator-libs-linux",
        "library_name": "passwd",
        "library_id": "cf7655b2bf914d67ac963f72b930f6bb",
        "api": 0,
        "patch": 4,
        "content_hash": "441ad5d92530c227a78e2207f5650f86a8b842134545e222514b3d6425ed9673"
    },
    {
        "charm_name": "operator-libs-linux",
        "library_name": "snap",
        "library_id": "05394e5893f94f2d90feb7cbe6b633cd",
        "api": 1,
        "patch": 12,
        "content_hash": "ab4190b96a2bddfa60caf1c14a74868f378b5243396e6e4a2b6604c6b9b10976"
    },
    {
        "charm_name": "operator-libs-linux",
        "library_name": "systemd",
        "library_id": "045b0d179f6b4514a8bb9b48aee9ebaf",
        "api": 1,
        "patch": 3,
        "content_hash": "56e0dad863d86816ef55ac1fae4f61a42d8d66a73d49620ea8d6768282480575"
    }
]
found lib file: ./lib/charms/operator_libs_linux/v0/apt.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v0/dnf.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v0/passwd.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v1/systemd.py. Parsing...
found lib file: ./lib/charms/operator_libs_linux/v2/snap.py. Parsing...
checking status for apt
remote lib: {"libName":"apt","version":0,"revision":11}
apt and apt are equal
checking status for dnf
remote lib: {"libName":"dnf","version":0,"revision":1}
dnf and dnf are equal
checking status for passwd
remote lib: {"libName":"passwd","version":0,"revision":4}
passwd and passwd are equal
checking status for systemd
remote lib: {"libName":"systemd","version":1,"revision":3}
systemd and systemd are equal
checking status for snap
remote lib: {"libName":"snap","version":1,"revision":12}
snap has a lower version or revision than snap
Errors found during lib comparison.
Nothing to update. Exiting...

### Additional context

_No response_

Enhancement Proposal

Hi,

We are running check-libraries on PRs to make sure our charms are using the latest shared libraries.
The approach with green/red labels works perfectly: example.

What is missing is a human protection to inform author about forgotten LIBPATCH bump on the shared library change: example.

What about to set red label: Libraries: bump LIBPATCH if author changes shared library and forget to bump it?
At the moment the action simply prints notice and sets green label Libraries: OK:

Library charms.mongodb.v0.helpers has local changes, cannot be updated.
Library charms.mongodb.v0.mongodb_backups has local changes, cannot be updated.
Library charms.mongodb.v0.mongodb_vm_legacy_provider has local changes, cannot be updated.

We have tried to use option fail-build, but it didn't fail on locally changed library.
Is it a broken option or it was designed for something else? Tnx!

Every time we open a PR we should publish the built charm to a Charmhub branch.

For example, if we open a PR-25 for the git branch track/1.4 the charm built in that PR should be pushed to the charmhub branch 1.4/edge/pr-25 everytime the CI of the PR is running.

This will allow for easier testing of PRs as we will just have to pull the build from charmhub. Additionally, it will test if the charm is publishable before merging the PR.

Enhancement Proposal

The upload charm action doesn't have an input for setting the Charmhub upload name (analogous to charmcraft upload --name). We want to upload the charm under a different name for testing.

      - name: Upload charm to charmhub
        uses: canonical/charming-actions/[email protected]
        with:
          name: ....

Enhancement Proposal

Currently, the juju-bundle implementation in the upload-bundle action changes directories to the specified path and looks for a bundle.yaml file as seen in here:

import * as core from '@actions/core';
import * as exec from '@actions/exec';

class Bundle {
  async publish(path: string, channel: string) {
    core.exportVariable('CHARMCRAFT_AUTH', core.getInput('credentials'));
    process.chdir(path);
    await exec.exec('juju-bundle', [
      'publish',
      '--destructive-mode',
      '--serial',
      '--release',
      channel,
    ]);
  }
}

export { Bundle };

This unnecessarily enforces bundle names which creates unnecessary rigidity to how bundle repos have to be structured. Not only do the bundles have to be named bundle.yaml, but they also will have to be in separate directories. This makes little sense when a path to a specific bundle can be given straight to juju-bundle using --bundle.