Rsync is open-source software that can be used to sync files and folders from a local computer to a remote one and vice versa. A notable feature of Rsync is the ability to transfer encrypted files using SSH and SSL. In addition, the file transfer is performed in a single thread, unlike other similar programs, which creates a separate thread for each file transfer. This increases the speed and removes the additional delays that become a problem when transferring a large number of small files.
Rsync is a very flexible network-enabled sync tool. Due to its ubiquity on Linux and Unix-like systems and its popularity as a system scripting tool, it is included by default with most Linux distributions.
For Microsoft Windows, cwRsync is a free distribution of the Rsync utility optimized for quick and easy installation on Microsoft Windows. You can download cwRsync here, however, we recommend checking for the latest version here. Once you’ve downloaded this setup file, run it. You should choose the defaults that the installer gives you, including the default install path: C:\Program Files\cwRsync
STRONGLY recommend installing the program to the default folder.
Features of Rsync:
- Ability to maintain synchronization of entire directory trees;
- You can save symbolic links, hard links, file owners and permissions, metadata, and creation time;
- Doesn’t require any special privileges;
- File transfer in one stream;
- Support for RSH, SSH as transport;
- Anonymous Rsync support.
By default, Rsync uses SSH transport. If you want to use the Rsync transport, you need to do some minimal configuration of the Rsync server. This will allow you to not only sync files to a remote server but also receive them from it.
First, create a config file with the following content:
vi /etc/rsyncd.conf
pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock log file = /var/log/rsync.log [share] path = /tmp/share/ hosts allow = 123.123.133.133 hosts deny = * list = true uid = root gid = root read only = false
Here we set the path to our folder for synchronization, allow access to the server only from the IP address (123.123.133.133) and deny all other connections. The uid and gid parameters specify the user and group from which the daemon will be started. It is better not to use root, but to specify the user nobody and give him the rights to the folder in which the Rsync directories will be synchronized.
The Rsync server configuration is completed, it remains to save the file, start the Rsync server and add it to startup:
systemctl start rsync systemctl enable rsync
Rsync syntax
The basic Rsync syntax is very simple and works similarly to ssh, scp, and cp.
rsync options source destination
The source and destination can be a remote or local directory. For example ssh, rsync, samba server, or local directory. Options specify additional options for Rsync.
Rsync options
Now let’s take a quick look at the Rsync options. Not all options are listed here. For more details see man Rsync:
-v – Display detailed information about the copying process;
-q – Minimum information;
-c – Checking checksums for files;
-a – Archiving mode, when all the attributes of the original files are preserved;
-R – Relative paths;
-b – Create a backup copy;
-u – Do not overwrite newer files;
-l – Copy symbolic links;
-h – Display numbers in human-readable format;
-L – Copy the contents of links;
-H – Copy hard links;
-W – Copy entire files;
-p – Preserve file permissions;
-g – Save the group;
-t – Keep modification time;
-x – Work only in this file system;
-e – Use another transport, for example, ssh;
-z – Compress files before transfer;
–delete – Delete files that are not in the source;
–exclude – Exclude files by pattern;
–include – Include files by pattern;
–recursive – Loop over directories recursively;
–no-recursive – Disable recursion;
–progress – Show the progress of the file transfer;
–stat – Show transmission statistics;
–version – The version of the utility.
Copy and sync files locally
Rsync allows you to sync files and folders on the same computer. Let’s first take a look at using Rsync to sync a file locally:
rsync -zvh anyfile /mnt/backup_dir/
By specifying the –progress parameter, you can see how many percent have already been copied and how much is still left:
rsync -zvh --progress anyfile /mnt/backup_dir/
Synchronizing folders on the local computer
Synchronizing folders by Rsync is as easy as syncing files:
rsync -zvh /home/user/documents /mnt/backup_dir/
If you want all file attributes such as modification and creation date to be preserved, you must use the -a option:
rsync -azvh /home/user/documents /mnt/backup_dir/
Synchronization with a remote server
It is not much more difficult to synchronize files with a remote server. Let’s copy the local documents folder to the remote server:
rsync -avz /home/user/documents/ [email protected]:/home/
By default, Rsync will try to use SSH transport. If you want to use the previously created Rsync server, you need to specify it explicitly:
rsync -avz /home/user/documents/ rsync://123.123.133.133:/home/
Similarly, you can synchronize files with Rsync from a remote server:
rsync -avz [email protected]3.133:/home/ /home/user/documents/
Synchronizing files over ssh
The -e option is used to set the connection protocol. When using SSH, all transmitted data is encrypted and transmitted over a secure channel, so that no one can intercept it. To use SSH, you need to know the user’s password on the system.
Synchronizing files from a remote server via ssh will look like this:
rsync -avzhe ssh [email protected]:/home/ /home/user/documents/
If you are using a different port for ssh, then you can specify it by this method:
rsync -avzhe "ssh -p 22" [email protected]:/home/ /home/user/documents/
Synchronizing some files in Rsync
The –include and –exclude options let you specify which files to sync and which to exclude. Options work not only with files but also with directories.
For example, let’s copy all files starting with the letter S:
rsync -avze ssh --include 'S*' --exclude '*' /home/user/documents/ [email protected]:/home/
Delete on sync
During synchronization, you can delete files that are not on the machine where Rsync synchronization is coming from, for this, use the –delete option.
For example:
rsync -avz --delete [email protected]:/home/ /home/user/documents/
If, before executing this command, you create a file in the folder that is not on the remote server, then it will be deleted.
Removing original files
It is possible to delete the source files after the synchronization with the remote server is complete:
rsync --remove-source-files -zvh [email protected]:/home/anyfile.tar /home/user/documents/
Thus, anyfile.tar file will be deleted after copying to /home/user/documents/ folder is complete.
Rsync simulation mode
If you are a beginner and haven’t used Rsync yet, you might want to see how the command works without actually doing anything in the filesystem. There is a –dry-run option for this. The command will only output all performed actions to the terminal, without making any real changes:
rsync --dry-run --remove-source-files -zvh [email protected]:/home/anyfile.tar /home/user/documents/
Limiting the transmission rate
You can limit your network bandwidth usage with the –bwlimit option:
rsync --bwlimit=100 -avzhe ssh [email protected]:/home/ /home/user/documents/
Rsync only syncs parts of a file, if you want to sync the whole file use the -W option:
rsync -zvhW [email protected]:/home/anyfile.tar /home/user/documents/anyfile.tar