Coder Social home page Coder Social logo

sarotate's People

Contributors

visorask avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

sarotate's Issues

yyq command not found?

Keep getting this after a fresh install of ubuntu and sarotate. I tried install "yq" but still same errro:

Try 'sleep --help' for more information.
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
sleep: invalid time interval โ€˜โ€™
Try 'sleep --help' for more information.
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found
./sarotate: line 16: yyq: command not found

Service account 0 is outside the minsa: 1 and maxsa: 50 set in the config file with no 0.json in SA folder

<14>May 11 11:01:31 sarotate: ====> BEGIN LOGGING
<14>May 11 11:01:31 sarotate:yyq_detect: Checking if version is 3.4.1.
<14>May 11 11:01:31 sarotate:yyq_detect: yyq is at version 3.4.1.
<14>May 11 11:01:31 sarotate:yyq_detect: Version check complete.
<11>May 11 11:01:32 sarotate:array_combine: Service account 0 is outside the minsa: 1 and maxsa: 50 set in the config file.
<11>May 11 11:01:32 sarotate:array_combine: Correct this and run the script again.
<11>May 11 11:01:32 sarotate:array_combine: The script will now exit.

this is the log I'm receiving.

why the script is trying to use the 0.json instead of 1.json?

I'm tried to make a workaround, copying the 1.json to 0.json and the script starts, but it does use the 2.json for all remotes, and not only for the first one in the list, contrary to what is set in config.yml.

Funny part is that I have another machine where the script it's working without a problem, so I have no idea why on this ubuntu linux is creating so many issues

Keep Track of Used SA's

I have noticed that it works well, until you either restart the process or the server, at which point it will go back to using the first SA in the list, which could in fact have hit a limit. Would be nice to keep a list of used ones and not repeat them till 24hrs has passed. Then allow that SA to go back into the rotation.

apprise notifcations for pushover

I was using this in the prev version but it doesn't appear to be working in the new version
Old Config File

  notif: "all"
  # Simple apprise command. Edit to taste.
  notif_cmd: "apprise -t SA-rotate  ``pover://code@code"```

New Config
apprise:
  # Apprise notifications.
  notif_cmd: "apprise -t SA-rotate  pover://code@code"
  # |discord|slack|off(default)|
  notif: off
  # Insert discord webhook information.
  # https://github.com/caronc/apprise/wiki/Notify_discord
  disc_id: "{WebhookID}/{WebhookToken}"
  # Insert webhook information.
  # https://github.com/caronc/apprise/wiki/Notify_slack
  slack_id: "{tokenA}/{tokenB}/{tokenC}"
# When filling these out you need to make sure the order of remote/port goes together or else
# the script will read wrong input/output and give errors.

Issues at first run

rclone mount is named "rc". Have 8 SA creds saved & named as 1 through 8. For this test I commented out all remotes but first one.

I also edited my rclone service file with:

--rc-no-auth \
--rc-addr=localhost:5572 \
$ ./sarotate -c config.yml

<14>Mar 20 14:23:07 sarotate: ====> BEGIN LOGGING
<14>Mar 20 14:23:07 sarotate:yyq_detect: Checking if version is 3.4.1.
<14>Mar 20 14:23:07 sarotate:yyq_detect: yyq is at version 3.4.1.
<14>Mar 20 14:23:07 sarotate:yyq_detect: Version check complete.
**********************************************
**                                          **
**          SARotate Version 3.3            **
**                                          **
**       Written by Lex and Visorak         **
**   https://github.com/Visorask/sarotate   **
**                                          **
**********************************************
<14>Mar 20 14:23:08 sarotate:main_rotate: SA Rotate is now initializing.
logger: invalid option -- '>'
Try 'logger --help' for more information.
<14>Mar 20 14:23:08 sarotate:log_rotate: Skipping log rotation: /opt/sarotate/logs/sarotate.log < 1048576
<14>Mar 20 14:23:08 sarotate:main_rotate: Switching remote rc to use service account 0.json for 15m.
2021/03/20 14:23:08 Failed to rc: connection failed: Post "http://127.0.0.1:5572/backend/command": dial tcp 127.0.0.1:5572: connect: connection refused
<14>Mar 20 14:23:08 sarotate:main_rotate: SARotate will now go idle for 15m.

Need help with setting up the config

First of all thanks a lot for the work on this project . Just trying it out for the very first time
At the moment , I don't need sa rotation for rclone mount . I primarily need it for all other rclone operations like copy ,sync etc

  1. So my first question was will i have to setup the config file bit differently to make copy and sync work with sarotate.

  2. Secondly i don't quite understand the port thing at the very bottom of the sarotate config
    Lets say i have 2 remotes named - library1 and library2 . And i need to copy from library1 to library2 , then i need to add them both in the remote option at bottom of config correct ? Now what's should be the port for these ?

  3. After the sarotate service is running well , how do i run the rclone copy command to make it use sarotate ?
    rclone copy library1: library2: --drive-server-side-across-configs running the copy command normally like this will utilise sarotate or i will have to make some changes to it ?

Confused regarding all this , can you please help regarding the same

Failed to rc: Failed to read rc response: 500 Internal Server Error

EDIT OMG, forgot about your warn. I'm reloading all configs to see whats going on.
Well mounts are but I still got the error.

Hi,

Thanks for the script but it seems I misunderstood it or my config is bad. I want to use sarotate to push up more than 750GB daily to my teamdrive.

Here are the logs:

2020-10-27:11:06:02 INFO    /opt/sarotate/sarotate:73 SARotate will now go idle for 15m.
2020-10-27:11:10:32 INFO    /opt/sarotate/sarotate:63 SA Rotate is now initializing these mounts, starting with /home/aerya/outils/safiles/1.json with an increment of 1.
2020-10-27:11:10:32 INFO    /opt/sarotate/sarotate:67 Switching remote g5crypt to use service account 1.json for 15m
Failed to rc: Failed to read rc response: 500 Internal Server Error: {
	"error": "command \"set\" failed: command not found",
	"input": {
		"command": "set",
		"fs": "g5crypt:",
		"opt": {
			"service_account_file": "/home/aerya/outils/safiles/1.json"
		}
	},
	"path": "backend/command",
	"status": 500
}
2020-10-27:11:10:32 INFO    /opt/sarotate/sarotate:67 Switching remote g6crypt to use service account 2.json for 15m
Failed to rc: Failed to read rc response: 500 Internal Server Error: {
	"error": "command \"set\" failed: command not found",
	"input": {
		"command": "set",
		"fs": "gHDO6crypt:",
		"opt": {
			"service_account_file": "/home/aerya/outils/safiles/2.json"
		}
	},
	"path": "backend/command",
	"status": 500
}
2020-10-27:11:10:32 INFO    /opt/sarotate/sarotate:73 SARotate will now go idle for 15m.

My mounts:

[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/root/.config/rclone/rclone.conf
KillMode=none
RestartSec=5
ExecStart=/usr/bin/rclone mount g5crypt: /home/aerya/mnt/mounts/g5crypt --allow-other --allow-non-empty --log-level INFO --rc --rc-addr :5572 --rc-no-auth --log-file /home/aerya/outils/logs/mounts/g5crypt.log --poll-interval 15s --umask 002 --user-agent=myuag5 --cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 50G --vfs-cache-max-age 1000h
ExecStop=/bin/fusermount -uz /home/aerya/mnt/mounts/g5crypt
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr 0.0.0.0:5572 _async=true
Restart=on-failure

[Install]
WantedBy=multi-user.target

[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/root/.config/rclone/rclone.conf
KillMode=none
RestartSec=5
ExecStart=/usr/bin/rclone mount g6crypt: /home/aerya/mnt/mounts/g6crypt --allow-other --allow-non-empty --log-level INFO --rc --rc-addr :5573 --rc-no-auth --log-file /home/aerya/outils/logs/mounts/g6crypt.log --poll-interval 15s --umask 002 --user-agent=myuag6 --cache-dir=/cache --vfs-cache-mode full --vfs-cache-max-size 50G --vfs-cache-max-age 1000h
ExecStop=/bin/fusermount -uz /home/aerya/mnt/mounts/g6crypt
ExecStartPost=/usr/bin/rclone rc vfs/refresh recursive=true --rc-addr 0.0.0.0:5573 _async=true
Restart=on-failure

[Install]
WantedBy=multi-user.target

And my config file for sarotate:

# General rclone variables.
rclone:
  # Default rclone config. Change this if you have a non-standard rclone config location.
  rclone_config: "root/.config/rclone/rclone.conf"
  # Match rclone rc mount user and pass. If you use --no-auth then leave as default.
  rc_user: "user"
  # Match rclone rc mount user and pass. If you use --no-auth then leave as default.
  rc_pass: "pass"
  # Default is localhost/127.0.0.1 but you can change this to match your IP for RC.
  ip: 127.0.0.1
# Global variables
global:
  # Minimum json file that you want to use.
  minjs: 1
  # Maximum json file that you want to use.
  maxjs: 20
  # Use 1 or 101.  101 will split API usage to separate projects, but needs many service accounts.
  nextjs: 1
  # Location of the directory with your service account files.
  json_dir: "/home/aerya/outils/safiles"
  # Time the script will sleep for until running again.
  sleeptime: 15m
# Log variables
log:
  # Directory for logs.
  log_dir: "/home/aerya/outils/logs/"
  # Name of log file.
  log_file: "/home/aerya/outils/logs/sarotate.log"
  # Set NOTIF="key" , "all" or "none". Key includes errors.
  notif: "none"
  # Simple apprise command. Edit to taste.
  notif_cmd: "apprise"
# When filling these out you need to make sure the order of remote/port goes together or else
# the script will read wrong input/output and give errors.
main:
  remote:
   - g5crypt
   - g6crypt
  port:
   - 5572
   - 5573

Any tip/advice would be appreciated!

Enhancement: work with rclone union remotes & remove sa from remote

Thanks for making this awesome utility!

Two enhancement requests for consideration:

  1. Support changing service account on underlying remote of a union remote which appears to be supported by rclone:
    https://forum.rclone.org/t/edit-service-account-while-mounted/15760/
    Specific post with command:
    https://forum.rclone.org/t/edit-service-account-while-mounted/15760/104
  2. Is there a way to remove the service account from the remote so it falls back to it's oauth key/secret/token, preferably a script we could add to execstop on the service to put the remote back to it's default state without any sa's attached or without the need to remount so our dir cache stays in tact?
  3. Thanks again and thanks for your consideration and time.
  4. Ed

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.