animmouse / setup-rclone

Setup/Install Rclone for GitHub Actions

Setup Rclone for GitHub Actions

Setup Rclone on GitHub Actions to use rclone.

This action installs Rclone for use in actions by installing it on tool cache using AnimMouse/tool-cache.

This action is implemented as a composite action.

Usage

To use rclone, run this action before rclone.

Encode the rclone.conf file in Base64 using this command base64 -w 0 rclone.conf and paste it to RCLONE_CONFIG secret.

Single command

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      
  - run: rclone copy source:sourcepath dest:destpath

For bare remote, use single quotes to stop the YAML parser from interpreting the remote as a key-value pair.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      
  - run: 'rclone copy source: dest:'

Multiple commands

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      
  - run: |
      rclone copy source:sourcepath dest:destpath
      rclone copy source: dest:

For bare remote in pipes, there is no need to use single quotes as YAML parser does not interpret the remote as a key-value pair.

Encrypted Rclone config

Paste your Rclone config pass to RCLONE_CONFIG_PASS secret.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      
  - run: rclone copy source:sourcepath dest:destpath
    env:
      RCLONE_CONFIG_PASS: ${{ secrets.RCLONE_CONFIG_PASS }}

Config-less operation

You can use Rclone without a config file by using command-line options or environment variables.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    
  - run: 'rclone lsd --http-url https://beta.rclone.org :http:'
    
  - run: 'rclone lsd :http:'
    env:
      RCLONE_HTTP_URL: https://beta.rclone.org

Plain text Rclone config

You can disable Base64 so that you can input the config file in plain text. This allows you to place the config file directly in YAML or get the config file from outputs.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: |
        [rclone-test-remote]
        type = http
        url = https://beta.rclone.org/test/
      disable_base64: true
      
  - run: 'rclone lsd rclone-test-remote:'

Service account file

To prevent the Rclone config file from becoming too large since you have a lot of remotes that uses the same service account file, use the AnimMouse/setup-rclone/service-account-file@v1 action to add the service account file inside the Rclone config directory.

Encode the service-account-file.json file in Base64 using this command base64 -w 0 service-account-file.json and paste it to SERVICE_ACCOUNT_FILE secret. And declare the service account file on the Rclone config like this service_account_file = $RCLONE_CONFIG_DIR/service-account-file.json.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      
  - name: Add Rclone service account file
    uses: AnimMouse/setup-rclone/service-account-file@v1
    with:
      service_account_filename: service-account-file.json
      service_account_file: ${{ secrets.SERVICE_ACCOUNT_FILE }}
      
  - run: rclone copy source:sourcepath dest:destpath

You can also declare multiple service account files; just run the AnimMouse/setup-rclone/service-account-file@v1 action multiple times, and use different service account filename and different secret name.

Update token

Some tokens in Rclone config have expiration, which means they have to be refreshed, or else they will expire and it will not work anymore. To prevent expiration, Rclone automatically refreshes the tokens as needed. To update those tokens in GitHub secrets, use the AnimMouse/setup-rclone/update-config@v1 action to update the Rclone config.

This requires a fine-grained personal access token that has read and write access to the secrets scope in the current repository to update the secret as the default GITHUB_TOKEN does not have access to the secrets scope.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      
  - run: rclone copy source:sourcepath dest:destpath
    
  - name: Update Rclone config
    uses: AnimMouse/setup-rclone/update-config@v1
    with:
      rclone_config_secret_name: RCLONE_CONFIG
      token: ${{ secrets.GH_PAT }}

Specific version

You can specify the version you want. By default, this action downloads the latest version if the version is not specified.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      version: v1.64.0

GitHub token

This action automatically uses a GitHub token in order to authenticate with the GitHub API and avoid rate limiting. You can also specify your own read-only fine-grained personal access token.

steps:
  - name: Setup Rclone
    uses: AnimMouse/setup-rclone@v1
    with:
      rclone_config: ${{ secrets.RCLONE_CONFIG }}
      token: ${{ secrets.GH_PAT }}

Similar actions

1. NiceLabs/rclone-action
2. wei/rclone
3. andreiio/rclone-action

setup-rclone's Issues

Don't force 3rd party dependencies

Hey,

We've noticed your scripts are using aria2c and 7z to download and extract the rclone zip.
This creates a dependency requirements for self-hosted runners that can be avoided.
An additional benefit of removing these dependencies is that your code will run on any windows or linux machine (or at least the majority of them)

Windows

Download

You can do something similar to this:

$ProgressPreference = 'SilentlyContinue'  # hide progress barr
Invoke-WebRequest -Uri "https://....." -OutFile "C:\path\to\aria2c"
$ProgressPreference = 'Continue'  # return progress bar

Unzip

Something like this should work in powershell:

Expand-Archive C:\a.zip -DestinationPath C:\a

Linux

Download

The majority of linux distributions have curl or wget pre-installed.
This is also available in ubuntu and probably is already available in the github cloud runners

Unzip

The majority of linux distributions have unzip installed.
This is also available in ubuntu and probably is already available in the github cloud runners

Add optional token input to avoid GitHub API rate limiting on macOS

Since macOS VMs use a few IP addresses, this action sometimes fail on macOS returning null in version variable, failing the download as the IP address got rate limited. actions/runner-images#602

API rate limit exceeded for IP. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

[ERROR] CUID#7 - Download aborted. URI=https://github.com/rclone/rclone/releases/download/null/rclone-null-osx-amd64.zip
Exception: [AbstractCommand.cc:351] errorCode=3 URI=https://github.com/rclone/rclone/releases/download/null/rclone-null-osx-amd64.zip
-> [HttpSkipResponseCommand.cc:219] errorCode=3 Resource not found

Fails with "jq: not found"

Run AnimMouse/setup-rclone@v1
  with:
    rclone_config: ***
Run $GITHUB_ACTION_PATH/scripts/download/Linux.sh
/__w/_actions/AnimMouse/setup-rclone/v1/scripts/download/Linux.sh: 3: jq: not found
Error: Process completed with exit code 127.

• Job:

runs-on: ubuntu-latest
container: python:3.10

• Step:

- name: Setup Rclone
   uses: AnimMouse/setup-rclone@v1
   with:
     rclone_config: ${{ secrets.RCLONE_CONFIG }}

Please update the action dependencies

The warning began to appear while running AnimMouse/setup-rclone@v1 action:

Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/cache/restore@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/

Thank you in advance

Latest version number and proper failure is not guaranteed

One of our workflow run failed with the following error:

[ERROR] CUID#7 - Download aborted. URI=https://github.com/rclone/rclone/releases/download/null/rclone-null-osx-amd64.zip
Exception: [AbstractCommand.cc:351] errorCode=3 URI=https://github.com/rclone/rclone/releases/download/null/rclone-null-osx-amd64.zip
-> [HttpSkipResponseCommand.cc:219] errorCode=3 Resource not found

It seems to be that the error came from here, where for some reason the latest version was determined as null:

setup-rclone/scripts/download/macOS.sh

Line 2 in 260c6f9

version=$(curl -s https://api.github.com/repos/rclone/rclone/releases/latest | jq -r '.tag_name')

Maybe some additional check/retry logic could be implemented in case the version is not properly returned, to make the run more resilient.

This step also didn't fail (which it should have), even though it failed to install the tool. Additional check could be implemented for that too.