天增的博客
首页
博客
  • 分布式解决方案
  • Java并发工具包
  • redis
  • LeetCode
  • 系统设计
  • JVM体系
Github (opens new window)
Rss (opens new window)
  • zh-CN
  • en-US
首页
博客
  • 分布式解决方案
  • Java并发工具包
  • redis
  • LeetCode
  • 系统设计
  • JVM体系
Github (opens new window)
Rss (opens new window)
  • zh-CN
  • en-US
  • posts
  • 后台使用Rclone进行同步
2022-07-21

后台使用Rclone进行同步

# 后台使用Rclone进行同步

Rclone (opens new window) 是一个杰出的、开源的文件同步工具。

支持了很多不同网盘的同步服务,然而,在使用Rclone进行同步的时候,像Dropbox或者Onedriver一样,在保存文件的时候进行无感知的同步,Github有个issue详细讨论了这个问题https://github.com/rclone/rclone/issues/249 (opens new window)

办法总比困难多,为了解决这个问题,通常的做法是使用Cron脚本,定时执行Sync的命令,以达到无感知同步的功能。

我们可以改善使用Cron同步文件的行为,linux中有监控文件改变的脚本inotifywait (opens new window) , 安装方式在 https://command-not-found.com/inotifywait (opens new window)

将inotifywait 注册成系统服务,这样即使在系统重启之后也能够继续保持同步。

‍

下面是脚本内容:

#!/bin/bash
## One-way, immediate, continuous, recursive, directory synchronization
##  to a remote Rclone URL. ( S3, SFTP, FTP, WebDAV, Dropbox, etc. )
## Optional desktop notifications on sync events or errors.
## Useful only for syncing a SMALL number of files (< 8192).
##  (See note in `man inotifywait` under `--recursive` about raising this limit.)
## Think use-case: Synchronize Traefik TLS certificates file (acme.json)
## Think use-case: Synchronize Keepass (.kdbx) database file immediately on save.
## Think use-case: Live edit source code and push to remote server

## This is NOT a backup tool!
## It will not help you if you delete your files or if they become corrupted.
## If you need a backup tool, check out https://blog.rymcg.tech/blog/linux/restic_backup

## Setup: Install `rclone` from package manager.
## Run: `rclone config` to setup the remote, including the full remote
##   subdirectory path to sync to.
## MAKE SURE that the remote (sub)directory is EMPTY
##   or else ALL CONTENTS WILL BE DELETED by rclone when it syncs.
## If unsure, add `--dry-run` to the RCLONE_CMD variable below,
##   to simulate what would be copied/deleted.
## Enable your user for Systemd Linger: sudo loginctl enable-linger $USER
## (Reference https://wiki.archlinux.org/title/Systemd/User#Automatic_start-up_of_systemd_user_instances)
## Copy this script any place on your filesystem, and make it executable: `chown +x sync.sh`
## Edit all the variables below, before running the script.
## Run: `./sync.sh systemd_setup` to create and enable systemd service.
## Run: `journalctl --user --unit rclone_sync.${RCLONE_REMOTE}` to view the logs.
## For desktop notifications, make sure you have installed a notification daemon (eg. dunst)

## Edit the variables below, according to your own environment:

# RCLONE_SYNC_PATH: The path to COPY FROM (files are not synced TO here):
RCLONE_SYNC_PATH="/home/user/dav-sync"

# RCLONE_REMOTE: The rclone remote name to synchronize with.
# Identical to one of the remote names listed via `rclone listremotes`.
# Make sure to include the final `:` in the remote name, which
#   indicates to sync/delete from the same (sub)directory as defined in the URL.
# (ALL CONTENTS of the remote are continuously DELETED
#  and replaced with the contents from RCLONE_SYNC_PATH)
RCLONE_REMOTE="webdav-remote:"

# RCLONE_CMD: The sync command and arguments:
RCLONE_CMD="rclone -v sync ${RCLONE_SYNC_PATH} ${RCLONE_REMOTE}"

# WATCH_EVENTS: The file events that inotifywait should watch for:
WATCH_EVENTS="modify,delete,create,move"

# SYNC_DELAY: Wait this many seconds after an event, before synchronizing:
SYNC_DELAY=5

# SYNC_INTERVAL: Wait this many seconds between forced synchronizations:
SYNC_INTERVAL=3600

# NOTIFY_ENABLE: Enable Desktop notifications
NOTIFY_ENABLE=true

# SYNC_SCRIPT: dynamic reference to the current script path
SYNC_SCRIPT=$(realpath $0)

notify() {
    MESSAGE=$1
    if test ${NOTIFY_ENABLE} = "true"; then
        notify-send "rclone ${RCLONE_REMOTE}" "${MESSAGE}"
    fi
}

rclone_sync() {
    set -x
    # Do initial sync immediately:
    notify "Startup"
    ${RCLONE_CMD}
    # Watch for file events and do continuous immediate syncing
    # and regular interval syncing:
    while [[ true ]] ; do
	inotifywait --recursive --timeout ${SYNC_INTERVAL} -e ${WATCH_EVENTS} \
		    ${RCLONE_SYNC_PATH} 2>/dev/null
	if [ $? -eq 0 ]; then
	    # File change detected, sync the files after waiting a few seconds:
	    sleep ${SYNC_DELAY} && ${RCLONE_CMD} && \
		notify "Synchronized new file changes"
	elif [ $? -eq 1 ]; then
	    # inotify error occured
	    notify "inotifywait error exit code 1"
        sleep 10
	elif [ $? -eq 2 ]; then
	    # Do the sync now even though no changes were detected:
	    ${RCLONE_CMD}
	fi
    done
}

systemd_setup() {
    set -x
    if loginctl show-user ${USER} | grep "Linger=no"; then
	    echo "User account does not allow systemd Linger."
	    echo "To enable lingering, run as root: loginctl enable-linger $USER"
	    echo "Then try running this command again."
	    exit 1
    fi
    mkdir -p ${HOME}/.config/systemd/user
    SERVICE_FILE=${HOME}/.config/systemd/user/rclone_sync.${RCLONE_REMOTE}.service
    if test -f ${SERVICE_FILE}; then
	    echo "Unit file already exists: ${SERVICE_FILE} - Not overwriting."
    else
	    cat <<EOF > ${SERVICE_FILE}
[Unit]
Description=rclone_sync ${RCLONE_REMOTE}

[Service]
ExecStart=${SYNC_SCRIPT}

[Install]
WantedBy=default.target
EOF
    fi
    systemctl --user daemon-reload
    systemctl --user enable --now rclone_sync.${RCLONE_REMOTE}
    systemctl --user status rclone_sync.${RCLONE_REMOTE}
    echo "You can watch the logs with this command:"
    echo "   journalctl --user --unit rclone_sync.${RCLONE_REMOTE}"
}

if test $# = 0; then
    rclone_sync
else
    CMD=$1; shift;
    ${CMD} $@
fi

‍

最近更新
01
以 root 身份启动 transmission-daemon
12-13
02
Debian系统安装qbittorrent-nox
12-09
03
LXC Debain12安装zerotier并实现局域网自动nat转发
07-29
更多文章>
Theme by Vdoing | Copyright © 2015-2024 天增 | 苏ICP备16037388号-1
  • 跟随系统
  • 浅色模式
  • 深色模式
  • 阅读模式